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ABSTRACT 


The  Department  of  Defense  (DoD)  spends  about  $15  billion  annually  on  depot  level 
maintenance.  About  60  percent  of  this  funding  is  provided  to  government  owned  and  operated 
depots.  In  light  of  defense  budget  downsizing,  it  has  become  more  critical  than  ever  that  depots 
are  run  in  the  most  efficient  manner  possible.  DoD  has  tried  to  adopt  a  “best  commercial 
practices”  approach  to  improve  efficiency  of  depot  maintenance.  A  key  focus  of  commercial 
practices  is  delivering  customer  satisfaction.  To  this  extent,  it  is  imperative  that  DoD  depots 
understand  and  properly  measure  their  customer’s  concerns  if  they  wish  to  improve  their 
performance.  An  adaptation  of  the  gaps  model,  developed  by  Parasuraman,  Zeithamal  and  Berry 
in  1985,  was  used  to  measure  the  current  customer  satisfaction  of  the  NADEP  NI  F/A-18  aircraft 
maintenance  program.  The  gaps  model  measures  differences  between  customer  expectations  and 
perceptions  of  performance  of  various  attributes,  and  ranks  the  attributes  by  importance.  A 
pretest  questionnaire  was  developed  and  sent  out  to  customers  of  NADEP  NI’s  F/A-18  aircraft 
maintenance  program  in  order  to  evaluate  alternative  measures  of  customer  satisfaction.  Through 
this  process,  a  tailored  set  of  customer  satisfaction  measures  was  developed  to  provide  better 
feedback  to  the  depot  management  team  and  improve  the  depot  maintenance  process. 
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1.  INTRODUCTION 


A.  WHAT  IS  CUSTOMER  SATISFACTION?  A  BRIEF  HISTORY 


Customer  satisfaction  is  the  customer's  perception  that  the  service  provider's 
performance  meets  or  exceeds  the  customer's  expectations.  Measuring  customer 
satisfaction  requires  assessing  both  customer  expectations  and  the  actual  and  perceived 
quality  of  service.  The  concept  of  customer  satisfaction  measurement  is  not  new.  It  has, 
however,  been  increasing  in  importance  as  a  cornerstone  of  a  successful  business 
philosophy.  This  trend  began  after  World  War  H,  when  W.  Edwards  Deming  went  to 
Japan  to  teach  quality.  J.M.  Juran,  who  also  went  to  Japan  after  the  war  to  help  Japanese 
industry  rebuild,  is  credited  with  linking  quality  to  customer  satisfaction  and  fitness  for 
use. 


Ironically,  the  Western  world  did  not  embrace  Deming  and  Juran's  concepts  until 
Japan  became  a  global  economic  threat  in  the  late  1970's  and  early  1980's.  By  that  time, 
Japan  was  producing  products  and  services  of  equal  or  higher  quality  than  Western 
manufacturers,  and  at  a  lower  cost.  Other  factors  were  also  contributing  to  a  general  rise 
in  international  competition.  These  were: 

*  improved  transportation  and  communication  networks 

*  a  reduction  in  trade  barriers 

*  universal  access  to  both  basic  and  advanced  industrial  know-how  and 
technology 

*  a  colossal  increase  in  manufacturing  output  by  countries  outside  North 
America 

*  new  types  of  international  technology  agreements 

*  the  ability  of  small  firms  to  compete  with  larger  multinationals  by  focusing  on 
niche  markets.  (Noori  1990) 
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This  influx  of  high  quality,  low  cost  goods  into  Western  markets  started  the 
growth  of  consumerism.  Customers  began  to  demand  quality,  reliable  goods  and  services 
from  Western  manufacturers.  Western  manufacturers  responded  for  fear  of  losing 
business  to  the  aggressive  growth  of  Japanese  manufacturers.  Still,  Western  producers 
and  service  providers  were  fairly  slow  in  shifting  towards  a  customer  oriented  approach  to 
quality.  A  major  milestone  was  reached  in  1979,  when  Philip  B.  Crosby  published  Quality 
is  Free.  Crosby  showed  how  the  cost  of  poor  quality  should  include  all  of  the  resultant 
costs  involved  in  not  doing  the  job  right  the  first  time,  as  defined  by  the  customer. 
Mmnstream  corporate  America  didn't  wholeheartedly  embrace  the  concept  until  the  early 
1990's.  "Now  with  most  everyone  on  the  bandwagon,  many  executives  and  management 
gurus  are  labeling  the  1990s  the  'Decade  of  the  Customer.’”  (Phillips  1990) 


B.  WHY  IS  CUSTOMER  SATISFACTION  IMPORTANT? 

No  one  would  argue  that  an  organization  should  produce  a  quality  product  or 
service.  However,  the  definition  of  quality  can  vary  depending  on  the  perspective  of  the 
individual  defining  it.  Quality  is  ultimately  defined  by  customers  as  the  degree  to  which 
they  are  satisfied  with  an  organization's  product  or  service. 

The  answer  to  why  customer  satisfaction  is  important  to  the  corporate  world  is 
easy:  profit.  A  case  study  of  the  Marriott  hotel  chain  found  that  "each  percentage  point 
increase  in  the  customer  satisfaction  measure-of-intent-to-return  was  worth  some  $50 
million  in  revenues."  (Connellan  1993)  Similarly,  a  case  study  of  IBM's  AS/400  computer 
manufacturing  site  learned  that  "a  one  percent  increase  in  customer  satisfaction  was  worth 
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$257  million  in  additional  revenues  over  the  ensuing  five  years."  (Connellan  1993) 
Especially  since  the  rise  of  global  competition  and  consumerism,  beginning  in  the  early 
1980's,  a  business  must  satisfy  customers  to  be  successful  in  the  long  run.  According  to 
Peterson  and  Wilson,  "customer  satisfaction  is  a  defensible  and  appropriate  company 
objective  -  the  glue  that  holds  various  corporate  functions  together  and  directs  corporate 
resource  allocation."  They  go  on  to  state  that  "virtually  all  company  activities,  programs, 
and  policies  should  be  evaluated  in  terms  of  their  contribution  to  satisfying  customers." 
(Peterson  &  Wilson  1992)  Correctly  measuring  customer  satisfaction  leads  to  more 
efficient  operations  which  can  reduce  costs  by  identifying  non-value  adding  tasks,  but  can 
also  increase  an  organization's  customer  base. 

The  same  principles  apply  in  the  public  sector.  Although  profit,  per  se,  is  not  the 
motive  for  most  government  agencies,  there  are  many  benefits  that  can  be  derived  from 
accurately  measuring  the  satisfaction  of  an  agency's  customers.  Some  of  the  more  obvious 
include; 

1)  Optimizing  resource  allocation  and  use  to  balance  customer  expectations 
with  departmental  mandates  and  available  resources  (people,  money  and 
time). 

2)  Identifying  opportunities  for  new  services  and  for  service  adjustment, 
which  could  mean  continuing,  discontinuing,  realigning  or  transferring 
services. 

3)  Improving  the  quality  and  effectiveness  of  government  services. 

4)  Determining  service  relevance  and  importance. 

5)  Setting  service  standards. 

6)  Providing  a  method  to  evaluate  employees  for  incentive  purposes. 

(Treasury  Board  of  Canada  Secretariat  1996) 
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The  Department  of  Defense  (DoD)  has  already  recognized  the  importance  of 

customer  satisfaction.  The  DoD  TQM  implementation  guide  states: 

The  customer  defines  the  purpose  of  the  organization  and  every  process 
within  it.  Success  means  striving  to  become  the  best  supplier  of  your 
particular  products  and  services  in  the  minds  of  those  customers.  To 
achieve  that  success,  your  organization  must  align  its  overriding  strategic 
vision  with  a  vision  of  customer  service  and  satisfaction.  (DoD  1990) 

Vice  President  Gore's  National  Performance  Review  also  focused  on  improving  the 

efiBciency  of  the  federal  government  by  focusing  on  its  customer  -  the  American  people. 


In  1993,  the  President  signed  Executive  Order  12862  -  Setting  Customer  Service 


Standards.  This  made  measuring  customer  satisfaction  the  law  for  all  executive 
departments  and  federal  government  agencies  that  provide  significant  services  directly  to 


the  public.  Requirements  include; 


1)  identifing  the  customers  who  are,  or  should  be,  served  by  the 
agency; 

2)  surveying  customers  to  determine  the  kind  and  quality  of  services 
they  want  and  their  level  of  satisfaction  with  existing  services; 

3)  posting  service  standards  and  measuring  against  them; 

4)  benchmarking  customer  service  performance  against  the  best  in 
business; 

5)  providing  customers  with  choices  in  both  the  sources  of  service  and 
the  means  of  delivery; 

6)  making  information,  service,  and  complaint  systems  easily 
accessible; 

7)  providing  means  to  address  customer  complaints.  (Clinton  1993) 


This  executive  order  drastically  changed  the  focus  of  many  government  agencies. 

Another  major  change,  which  affects  all  government  agencies,  not  just  those 
providing  significant  services  directly  to  the  public,  was  the  Government  Performance  and 
Results  Act,  passed  by  Congress  in  1993.  Federal  agencies  are  now  required  to  "plan 
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strategically,  develop  goals  that  are  outcome-focused,  consult  with  their  customers  when 
developing  strategic  plans  and  develop  performance  plans  that  look  to  intended  results, 
not  just  inputs  and  outputs."  (Federal  Quality  Institute  1993) 

C.  PURPOSE  -  THESIS  STATEMENT 

Due  to  the  requirements  mentioned  above,  and  at  the  request  of  the  Naval 
Aviation  Depot  (NADEP),  North  Island  (NI),  CA,  we  conducted  research  to  evaluate  the 
current  and  alternative  methods  of  measuring  customer  satisfaction  ofF/A-18  aircraft 
maintenance  performed  at  NADEP  NI.  The  goal  of  our  research  was  to  propose  a 
tailored  set  of  customer  service  measures  that  would  provide  better  feedback  to  the  depot 
management  team  and  help  improve  the  depot  maintenance  process.  This  analysis 
examines  the  customer  service  concerns  of  the  depot's  different  external  customers, 
including  squadrons,  wings,  type  commanders  and  the  Naval  Air  Systems  Command. 
Although  designed  specifically  to  benefit  NADEP  NI,  it  is  our  hope  that  this  research  will 
benefit  other  DoD  depot  maintenance  activities  and  government  service  agencies  in 
general. 

DoD  spends  about  $15  billion  annually  on  depot  level  maintenance.  About  60 
percent  of  this  funding  is  provided  to  government  owned  and  operated  depots.  In  light  of 
defense  budget  downsizing,  it  has  become  more  critical  than  ever  to  run  depots  in  the 
most  efficient  manner  possible.  The  General  Accounting  Office  has  recently  submitted 
several  reports  criticizing  DoD  for  inefficiently  managing  organic  depots. 

DoD  has  tried  to  adopt  a  "best  commercial  practices"  approach  to  improve 
efficiency  of  depot  maintenance.  A  key  focus  of  commercial  practices  is  delivering 
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customer  satisfaction.  To  this  extent,  it  is  imperative  that  DoD  depots  understand  and 
properly  measure  their  customer's  concerns  if  they  wish  to  improve  their  performance. 

D.  METHODOLOGY 

The  methodology  used  in  this  research  includes  the  following  steps. 

1 .  We  conducted  a  thorough  review  of  the  current  customer  satisfaction 
measurement  systems  at  NADEP  NI.  This  was  done  by  survey,  questionnaires,  and 
persormel  interviews  with  depot  persoimel.  The  current  system  of  measuring  customer 
satisfaction  at  NADEP  NI  is  described  in  Chapter  II. 

2.  We  conducted  a  literature  search  of  books,  magazine  articles,  CD-ROM 
systems,  internet  sources,  and  other  library  information  resources  that  dealt  with  customer 
satisfaction.  The  results  of  this  research  can  be  found  in  Chapter  III. 

3.  We  examined  customer  satisfaction  measurement  systems  currently  in  use  in 
the  private  and  public  sector.  We  identified  successful  examples  and  lessons  learned 
from  both  areas.  The  results  of  this  research  can  be  found  in  Chapter  III. 

4.  We  developed  a  pretest  questionnaire  to  be  sent  to  the  external  customers  of 
NADEP  NI's  F/A-18  aircraft  depot  maintenance  program  to  identify  their  expectations, 
importance  factors,  and  perceptions  of  performance  of  depot  maintenance.  The 
development  of  this  questionnaire  can  be  found  in  Chapter  IV.  For  the  purpose  of  this 
research  we  consider  the  depot's  external  customers  to  be  squadrons,  wings,  type 
commanders  and  the  Naval  Air  Systems  Command.  The  results  and  analysis  of  this  pretest 
questionnaire  can  be  found  in  Chapter  V. 
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5.  We  examined  the  customer  service  issues  in  the  move  of  F/A-18  depot 
maintenance  from  the  Navy  to  the  Air  Force,  and  back  to  the  Navy.  We  concentrated  on 
the  lessons  learned  and  how  an  accurate  measuring  system  of  customer  satisfaction  could 
have  surfaced  problems  with  customer  expectations.  The  results  of  this  research  can  be 
found  in  Chapter  II. 

6.  Finally,  we  prepared  a  recommended  set  of  tailored  customer  satisfaction 
measures  for  NADEP  NI's  F/A-18  aircraft  maintenance  program  with  an  implementation 
strategy.  This  recommendation  can  be  found  in  Chapter  VI. 

E.  RESEARCH  QUESTIONS 

The  folloAving  questions  guided  this  research  effort: 

1 .  How  is  the  concept  of  customer  satisfaction  relevant  to  the  DoD  depots? 

2.  What  current  systems  are  in  place  to  measure  customer  satisfaction  of  depot 
services? 

3 .  What  do  these  systems  truly  measure? 

4.  How  is  the  information  generated  by  current  measures  used  and  integrated  into 
depot  management  and  decision  making? 

5.  How  does  customer  satisfaction  information  flow  to  different  levels  of  the 
depot's  management  and  work  force?  Is  the  information  flow  timely? 

6.  How  is  customer  satisfaction  measured  for  similar  services  provided  in  the 
private  and  public  sector? 

7.  How  do  different  customers  (squadron,  wing,  type  commander,  systems 
command)  measure  depot  performance? 
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8.  Are  there  factors  limiting  the  depot's  responsiveness  to  customer  demands? 

9.  How  did  customer  satisfaction  impact  the  decision  to  change  the  venue  for 
F/A-18  depot  level  maintenance  from  the  Air  Force  back  to  NADEP  NI? 

10.  Are  there  alternative  methods  of  measuring  customer  satisfaction  that  would 
better  serve  the  depot  process? 

1 1 .  How  could  these  alternative  measures  be  best  implemented  at  NADEP  NI? 

12.  What  other  elements  are  required  to  establish  an  effective  feedback  loop 
connecting  the  depot  and  its  customers? 

F.  KEY  TERMS 

The  following  terms  are  defined  with  regard  to  customer  satisfaction  and 
measuring  quality  in  service  organizations.  This  list  provides  the  reader  with  a  point  of 
reference  in  understanding  our  research. 

1.  Quality  -  Fitness  for  use  as  defined  by  the  customer  or  "how  well  the  product 
or  service  performs  its  intended  function. "  (Hodgkiss  &  Casipit  1994) 

2.  Expected  Quality  -  What  the  customer  assumes  will  be  received  from  the 
product  (or  service)  as  a  reflection  of  the  customer's  needs.  (Hodgkiss  &  Casipit  1994 

3.  Perceived  Quality  -  The  customer's  measure  of  satisfaction  in  the  product,  the 
"feel"  for  its  quality.  (Hodgkiss  &  Casipit  1994) 

4.  Measure  -  "The  act  or  process  of  ascertaining  the  extent,  dimensions, 
quantity  of  something,  especially  by  comparison  with  a  standard."  (Random  House  1992) 

5.  Benchmarking  -  "Selecting  a  demonstrated  standard  of  performance  that 
represents  the  very  best  performance  for  a  process  or  activity."  (Heizer  &  Render  1996) 
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6.  External  Customer  -  The  customers  or  users  outside  of  the  provider’s 
organization. 

7.  Internal  Customer  -  "The  individual  or  department  within  the  organization 
that  receives  the  output  of  another  individual  or  department  within  the  organization." 
(Hodgkiss  &  Casipit  1994) 

8.  Metric  -  "A  measurement  made  over  time,  which  communicates  vital 
information  about  the  quality  of  a  process,  activity,  or  resource."  (Random  House  1992) 

9.  Quality  circle  -  "A  group  of  employees  meeting  regularly  with  a  facilitator  to 
solve  work-related  problems  in  their  work  area;  initiated  by  the  Japanese  in  the  1970's." 
(Heizer  &  Render  1996) 

10.  Quality  loss  function  -  "A  mathematical  function  that  identifies  all  costs 
connected  with  poor  quality  and  shows  how  these  costs  increase  as  product  or  service 
quality  moves  [away]  from  what  the  customer  wants."  (Heizer  &  Render  1996) 

11.  Tailoring  -  "Making  sure  that  the  element  measured  relates  to  specific 
information  needs,  that  it  is  measurable  and  that  the  information  is  meaningful."  (Heizer  & 
Render  1996) 

12.  Total  Quality  Management  (TQM)  -  "Management  of  an  entire 
organization  so  that  it  excels  in  all  aspects  of  products  and  services  that  are  important  to 
the  customer."  (Heizer  &  Render  1996) 

13.  Trend  -  "The  general  course  or  prevailing  tendency."  (Random  House  1992) 

14.  Target  value  -  "A  philosophy  of  continuous  improvement  to  produce 
products  or  services  that  are  exactly  what  the  customer  wants."  (Heizer  &  Render  1996) 
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15.  Efllciency  -  "The  ratio  of  service  quantity  output  to  the  amount  of  input 
required  to  produce  it."  (McLay  1992) 

16.  Effectiveness  -  "The  degree  to  which  the  intended  public  purposes  of  a 
service  or  activity  is  being  met."  (McLay  1992) 

17.  Productivity  -  "The  enhancement  to  the  production  process  that  results  in  a 
favorable  comparison  of  the  quantity  resources  employed  (inputs)  to  the  quantity  of  goods 
and  services  produced  (outputs)."  (McLay  1992) 

G.  ASSUMPTIONS 

The  first  major  assumption  of  this  research  is  that  NADEP  NI  has  a  mature  and 
effective  continuous  quality  improvement  program.  This  is  a  necessary  foundation  for 
effectively  using  the  data  produced  by  a  customer  satisfaction  measurement  system.  A 
second  assumption  is  that  both  NADEP  NI  and  its  external  customers  are  committed  to 
improving  the  effectiveness  of  depot  operations.  This  implies  that  all  organizations 
involved  are  willing  to  commit  the  necessary  resources  to  support  a  customer  satisfaction 
measurement  program  that  is  part  of  a  continuous  quality  improvement  effort.  The  third 
major  assumption  is  that  NADEP  NI  will  continue  to  verify  and  modify  the  recommended 
customer  satisfaction  program  to  ensure  that  it  remains  an  effective  tool  for  quality 
improvement.  A  final  assumption  is  that  NADEP  NI  truly  wants  to  measure  their 
customer's  satisfaction  with  NADEP's  products  and  services.  Measuring  customer 
satisfaction  is  often  “looking  for  bad  news.”  The  organization  must  view  customer 
complaints  as  invaluable  data  and  golden  opportunities  to  improve. 
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H.  CHAPTER  SUMMARY 


More  and  more,  private  industry  is  measuring  quality.  Companies  are  concluding 
that  if  they  can't  measure  it,  they  can't  manage  it  and,  consequently,  can't  improve  it.  The 
best  performing  organizations  are  allowing  customer  expectations  to  drive  their  quality 
initiative.  They  recognize  that  customers  define  quality  by  judging  their  products  and 
services  relative  to  competitors.  Organizations  that  constantly  measure  themselves  in 
relation  to  competitors  (benchmarking)  are  able  to  quickly  capitalize  on  their  emerging 
strengths  and  address  weaknesses  before  they  become  problems.  Continuous  quality 
improvement  begins  by  identifying  customer  expectations  for  all  key  interactions  between 
customers  and  the  organization. 

Although  government  agencies  are  not  driven  to  produce  a  profit,  their  survival 
can  hinge  on  the  quality  of  service  they  provide.  The  downsizing  budget  has  forced  the 
government  to  examine  its  effectiveness  in  a  hard  light.  Accurately  measuring  customer 
satisfaction  has  many  benefits  for  government  organizations  such  as:  optimizing  resource 
allocation  and  use  to  balance  customer  expectations  with  departmental  mandates  and 
available  resources  (people,  money  and  time);  identifying  opportunities  for  new  services 
and  for  service  adjustment,  which  could  mean  continuing,  discontinuing,  realigning  or 
transferring  services;  and  improving  the  quality  and  effectiveness  of  services. 
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n.  CURRENT  MEASURES  OF  CUSTOMER  SATISFACTION  AT  NADEP 


NORTH  ISLAND 

A.  NADEP  NORTH  ISLAND  AND  THE  F/A-18  MAINTENANCE  PROGRAM 
Naval  Aviation  Depot  North  Island  (NADEP  NI)  is  one  of  three  NADEPs 
providing  organic  depot  level  maintenance  and  engineering  support  to  Navy  and  Marine 
Corps  aircraft.  NADEP  NI  currently  employs  over  3500  workers  with  an  average  of  19 
years  of  experience.  (NADEP  fact  sheet)  During  the  recent  Base  Realignment  and 
Closure  (BRAC)  process,  the  Navy's  organic  depot  workload  was  redistributed  among  the 
three  remaining  NADEPs.  Part  of  this  redistribution  was  designed  to  assign  a  particular 
NADEP  as  the  "single  support  site"  for  each  aircraft  type  in  the  Navy/Marine  Corps 
inventory.  NADEP  NI  was  chosen  to  be  the  single  site  for  organic  depot  support  of  Navy 
and  Marine  Corps  F/A-18  aircraft.  In  practice,  however,  NADEP  Jacksonville,  FL 
(NADEP  JAX)  still  performs  some  F/A-18  maintenance. 

Within  NADEP  NI,  F/A-18  related  workload  falls  under  the  responsibility  of  the 
depot's  F/A-18  Program  Management  Team  Office  (PMTO),  with  the  exception  of 
component  overhaul  and  repair.  NADEP  NI  is  a  matrix  "competency  aligned" 
organization.  In  other  words,  only  a  small  number  of  NADEP  employees  performing  F/A- 
18  work  report  directly  to  the  PMTO.  Most  report  to  various  "competency  managers" 
who  assign  them  to  work  on  specific  programs.  A  diagram  of  NADEP  NI's  management 
structure  can  be  seen  in  Figure  2-1 . 

Several  F/A-18  workload  elements  are  controlled  by  NADEP's  F/A-18  PMTO. 
The  largest  of  these,  in  terms  of  cost  and  manhours  consumed,  is  a  scheduled  maintenance 
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Fiffure  2-1.  NADEP  North  Island  Competency  Aligned  Organization 


program  called  the  Modification,  Corrosion  and  Paint  Program  (MCAPP).  In  FY  97,  62 
F/A-18  aircraft  belonging  to  both  Pacific  and  Atlantic  fleet  type  commanders,  and  assigned 
to  various  U.S.  Navy  and  Marine  Corps  squadrons,  are  planned  to  complete  the  MCAPP 
process  at  NADEP  NI.  In  addition  to  MCAPP,  NADEP  NI  also  performs  in-service 
repair  (ISR)  of  damaged  F/A-18  aircraft  and  both  on-site  aircraft  repairs  and  depot  level 
modifications  through  field  teams. 

NADEP  NI  has  permanent  field  teams  located  at  Naval  Air  Station  (NAS) 
Lemoore,  CA,  Marine  Corps  Air  Station  (MCAS)  El  Toro,  CA  and  NAS  Fallon,  NV. 
Some  F/A-18  depot  level  maintenance  at  NAS  Cecil  Field,  FL  is  performed  by  field  teams 
from  NADEP  Jacksonville,  FL  due  to  their  close  proximity.  These  field  teams  perform 
three  important  functions.  First,  they  incorporate  a  variety  of  depot  level  modifications 
into  aircraft  at  the  customer  site.  This  process  is  usually  referred  to  as  "drive-in 
modification"  (DIM).  The  second  function  is  in-service  repair  of  damaged  aircraft.  The 
third  function  is  Paint  and  Corrosion  Evaluation  (PACE)  inspections.  These  inspections 
determine  if  an  aircraft  requires  MCAPP  .  Temporary  field  teams  are  organized  and 
dispatched  to  other  F/A-18  operational  sites,  as  required,  including  deployed  aircraft 
carriers. 

B.  THE  F/A-18  MAINTENANCE  CONCEPT  AND  MCAPP 

The  F/A-18  maintenance  concept  emphasizes  "on  condition"  maintenance.  Rather 
than  perform  scheduled  maintenance  tasks  at  a  particular  flight  hour  or  calender  milestone, 
on  condition  maintenance  is  performed  as  needed,  i.e.  -  when  the  aircraft  or  an  aircraft 
sub-system  has  reached  a  degraded  material  condition.  This  concept  extends  to  depot 


15 


level  maintenance  as  well.  Two  important  factors  drove  the  F/A-18  aircraft  to  this 
maintenance  concept.  The  first  is  cost.  In  the  long  run,  a  well  designed  on  condition 
maintenance  program  will  reduce  life-cycle  support  costs  for  the  aircraft  compared  to  a 
scheduled  maintenance  program.  Unnecessary  maintenance  will  be  minimized.  The 
second  factor  is  related  to  the  structural  design  of  the  F/A-18  aircraft.  Certain  areas  of  the 
aircraft  were  not  designed  to  be  disassembled  and  reassembled  without  compromising  the 
aircraft's  strength  and  fatigue  life.  Unnecessary  depot  level  maintenance  will  shorten  the 
aircraft's  service  life. 

The  F/A-18  aircraft's  on  condition  maintenance  philosophy  required  a  different 
approach  to  depot  maintenance  than  used  with  other  naval  aircraft.  The  resulting  unique 
depot  maintenance  program  is  MCAPP,  which  is  described  in  NAVAIR  F/A-18  MCAPP 
dated  3 1  Jan  1996.  Aircraft  are  flown  to  the  depot  to  go  through  MCAPP,  which 
currently  takes  an  average  of  108  calender  days  at  NADEP  NI.  While  in  MCAPP,  a 
variety  of  technical  directives  (TDs)  and  modifications  are  incorporated  into  the  aircraft 
depending  on  the  particular  aircraft  model  (there  are  four  different  F/A-18  models;  A,  B, 

C  and  D  models)  and  aircraft  lot  number  (current  production  aircraft  are  Lot  18).  Specific 
corrosion  prone  areas  of  the  aircraft  are  "opened  up,"  any  corrosion  damage  treated 
and/or  repaired,  and  the  aircraft's  paint  system  is  restored.  During  MCAPP  or  ISR,  the 
depot  does  not  fix  minor  discrepancies  found  which  are  not  included  in  the  MCAPP 
specification,  unless  they  affect  the  aircraft's  airworthiness.  To  determine  when  an  F/A-18 
aircraft  requires  MCAPP,  the  aircraft  undergoes  an  inspection  called  the  Paint  and 
Corrosion  Evaluation  (PACE).  The  content  and  conduct  of  this  inspection  is  governed  by 
a  NADEP  NI  Local  Engineering  Specification  (LES).  An  aircraft  gets  its  first  PACE 
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inspection  after  about  four  years  in  service.  If  the  aircraft  passes  the  PACE  inspection 
(score  of  45  points  or  lower  on  the  PACE  Corrosion  Evaluation  Discrepancy/History 
Report  form)  the  aircraft  remains  in  service  for  another  12  months,  at  which  time  it 
receives  another  PACE  inspection.  The  point  score  of  the  next  PACE  inspection  is  added 
to  the  scores  of  the  aircraft's  previous  PACE  inspections.  Once  the  score  exceeds  45 
points,  the  aircraft  is  scheduled  by  the  type  commander  for  MCAPP  induction.  PACE 
inspections  are  performed  by  NADEP  personnel  at  the  squadron  site.  While  NADEP  M 
performs  PACE  inspections  on  west  coast  based  F/A-18s,  NADEP  JAX  has  historically 
performed  P ACE  inspections  on  east  coast  based  aircraft.  This  arrangement  has  reduced 
consistency  in  judging  aircraft  material  condition.  NADEP  NI  hopes  to  perform  all  future 
PACE  inspections,  to  improve  consistency.  Also  towards  this  end,  most  NADEP  NI 
PACE  inspections  at  the  Navy’s  largest  F/A-18  base  (NAS  Lemoore,  CA)  are  performed 
by  the  same  individual. 

Reporting  custodians  (squadrons)  are  notified  by  the  type  command  staffs  of 
MCAPP  induction  dates.  Prior  to  sending  the  aircraft  to  depot  for  MCAPP,  the  squadron 
sends  an  MCAPP  Special  Work  Request  message  to  the  depot  identifying  required  depot 
level  TDs  and  another  items  requiring  special  rework  not  covered  by  the  standard  MCAPP 
work  specification.  In  practice,  though,  few  special  rework  items  are  requested  by 
squadrons. 

When  the  aircraft  is  about  mid-way  through  MCAPP,  NADEP  NI  sends  an 
MCAPP  Evaluation  Report  message  to  NAVAIR  and  the  type  commander,  with  an 
information  copy  to  the  type  wing  and  squadron.  This  message  identifies  which  TDs  will 
actually  be  incorporated  into  the  aircraft  and  explains  why  any  requested  TDs  or  special 
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rework  items  will  not  be  performed.  This  message  also  identifies  whether  the  number  of 
manhours  to  perform  the  MCAPP  work  on  the  aircraft  falls  within  the  depot's 
preestablished  norms,  or  whether  additional  manhours  need  to  be  approved  by  the  type 
command  staff. 

Upon  completing  MCAPP,  the  aircraft  usually  returns  to  the  same  squadron 
(greater  than  90  percent  of  the  time).  This  is  not  the  case  with  some  other  type  aircraft 
scheduled  depot  level  maintenance  (SDLM)  programs,  and  results  largely  fi-om  the  short 
MCAPP  turnaround  time  (relative  to  other  type  aircraft  SDLM  programs).  This  is  an 
important  advantage  for  the  F/A-18  community  for  two  reasons.  First,  there  are  a  great 
number  of  configuration  differences  between  different  aircraft  lot  numbers.  This  makes  it 
difficult  to  move  aircraft  around  between  squadrons,  for  aircrew  training  and  support 
reasons.  The  second  advantage  of  getting  the  same  aircraft  back  from  MCAPP  is  that  the 
squadron  can  better  assess  the  quality  of  work  done  by  the  depot  and  gauge  any  readiness 
or  performance  improvement. 

Aircraft  are  flown  back  to  the  squadron's  location  by  one  of  the  depot's  test  pilots. 
Though  these  pilots  interact  directly  with  the  squadron's  maintenance  personnel  and 
provide  NADEP  some  customer  feedback,  they  normally  return  to  the  depot  before  the 
squadron  completes  their  aircraft  acceptance  inspections.  On  its  return  to  the  squadron, 
the  aircraft  is  required  to  undergo  an  acceptance  inspection  and  a  functional  check  flight 
(FCF).  This  acceptance  process  normally  takes  about  3  working  days,  after  which  the 
aircraft  can  usually  be  scheduled  for  normal  squadron  use.  NADEP  NI  occasionally  has 
repot  representatives  remain  with  the  squadron  through  the  entire  acceptance/FCF 
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process,  to  better  understand  what  the  squadron  must  do  to  return  the  aircraft  to  regular 
flight  status. 

After  completing  the  acceptance/FCF  process,  the  squadron  submits  an  Aircraft 
Discrepancy  Report  (ADR)  to  the  depot,  even  if  no  discrepancies  are  noted  by  the 
squadron.  This  reporting  requirement,  spelled  out  in  OPNAVINST  4790.2F,  uses 
Standard  Form  368,  the  form  used  for  Category  2  Quality  Deficiency  Reports. 

Slots  for  MCAPP  induction  are  controlled  by  the  staffs  of  the  two  type 
commanders,  Commander  U.S.  Air  Forces  Pacific  Fleet  (COMNAVAJRPAC)  and 
Atlantic  Fleet  (COMNAVAIRLANT).  Currently  each  type  command  is  allocated  eight 
MCAPP  slots  per  quarter.  This  allocation  is  a  function  of  estimated  needs  and  available 
funds  for  depot  maintenance,  as  negotiated  between  the  type  commanders  and  NAVAIR 
Code  6.0  (AIR  6.0).  All  of  the  commands  involved  in  the  F/A-18  depot  maintenance 
process,  above  the  squadron  level,  gather  quarterly  at  different  sites  for  the  F/A-18 
Modification  Management  Meeting.  At  these  meetings,  F/A-18  issues  are  discussed  and 
negotiated,  including  MCAPP  slots,  MCAPP  content,  drive-in  modification  program 
content,  and  future  depot  maintenance  requirements. 

C.  THE  IMPORTANCE  OF  PLEASING  CUSTOMERS  AT  NADEP  NI 
NADEP  Mi's  mission  statement  begins  with  the  statement: 

As  a  fiill-service,  world  class  depot,  we  will  continue  to  excel  at  our 
principle  product  -  diverse  support  -  providing  our  customers  with  the 
highest  quality  at  the  best  value. 
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The  second  of  the  depot's  nine  Guiding  Principles  is: 

Customers  -  We  are  sensitive  and  responsive  to  the  needs  of  both  our 
internal  and  external  customers.  Our  existence  depends  on  their 
satisfaction.  Success  is  when  our  customers  brag  about  us. 

These  words  are  much  more  than  a  slogan  among  NADEP  NTs  employees,  especially 
those  connected  with  the  F/A-18  program.  These  depot  employees  know  the 
consequences  of  not  performing  up  to  customer  expectations. 

To  drive  down  costs  and  improve  schedule  performance,  the  Navy  competed  its 
F/A-18  MCAPP  workload  in  1992.  The  competition  was  open  to  both  public  and  private 
offerers.  If  all  options  of  the  proposed  4  year  contract  were  exercised,  the  total  workload 
was  valued  at  about  $61  million.  (GAO  96-31)  Bids  were  received  from  two  private 
contractors,  NADEP  NI  and  the  Air  Force's  Ogden  Air  Logistics  Center  in  Ogden,  Utah. 
The  Air  Force's  bid  was  significantly  lower  than  the  others  and  they  were  awarded  the 
contract.  The  Air  Force  received  their  first  F/A-18  for  MCAPP  in  December  1993.  The 
Air  Force  wanted  to  be  designated  as  the  single  site  for  F/A-18  MCAPP  but  the  Navy 
decided  (with  Office  of  the  Secretary  of  Defense  approval)  that  it  needed  to  maintain  a 
core  F/A-18  repair  capability  at  a  Navy  depot.  Over  the  next  1 1  months,  the  Air  Force 
inducted  36  F/A-18s  for  MCAPP  and  NADEP  NI  inducted  34.  MCAPPs  were  then  being 
performed  by  two  DOD  depots,  in  head  to  head  competition. 

As  a  result  of  losing  a  major  portion  of  its  F/A-18  workload,  NADEP  NI 
vigorously  examined  their  processes  and  reengineered  them  to  reduce  cost  and  turnaround 
time.  The  improvements  at  NADEP  NI  were  tremendous.  NADEP  NI's  average 
turnaround  time  for  aircraft  completing  MCAPP  in  FY  1994  was  269  days.  By  early 
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FY  1995,  NADEP  MI  was  consistently  completing  MCAPP  in  under  110  days.  The  Air 
Force  had  bid  to  complete  MCAPPs  in  143  days.  They  delivered  their  first  aircraft  on 
time,  but  all  subsequent  aircraft  were  delivered  from  17  to  over  200  days  late.  The  Air 
Force  felt  the  Navy  was  responsible  for  most  of  the  schedule  delays,  as  they  depended  on 
the  Navy  to  provide  parts  and  engineering  dispositions.  Regardless,  the  Navy  decided  that 
NADEP  NI's  schedule  performance  was  better  and  that  there  were  economic  advantages 
to  having  only  one  site  do  all  the  MCAPPs.  In  December  1994,  the  Navy  chose  not  to 
exercise  the  second  year  contract  option  and  consolidated  all  MCAPP  workload  back  at 
NADEP  NI.  (GAO  96-31) 

D.  NADEP  NX'S  EXTERNAL  CUSTOMERS 

Typical  of  many  large  government  organizations,  NADEP  NI  has  a  variety  of 
external  customers.  Just  within  NADEP's  F/A-18  Program  there  are  four  different  types 
of  external  customers.  These  are  the  squadrons,  the  wing  staffs,  the  type  commander 
staffs,  and  the  Naval  Air  Systems  Command  (AIR  6.0)  personnel.  Though  all  of  these 
customers  share  some  general  concerns  about  cost,  schedule  and  quality,  they  have  unique 
perspectives,  concerns  and  interactions  with  the  depot.  One  of  the  main  purposes  of  this 
thesis  is  to  differentiate  the  factors  that  govern  satisfaction  for  each  of  the  four  types  of 
customers  and  to  establish  their  relative  importance  for  each  customer. 

The  Navy  and  Marine  Corps  F/A-18  squadrons  who  "own"  the  aircraft  are  the 
depot's  most  obvious  customers.  While  researching  this  thesis  at  NADEP  NI,  and  talking 
with  a  cross  section  of  depot  employees,  we  were  told  by  most  that  the  squadrons,  or  "the 
fleet,"  were  the  depot's  most  important  customer.  Additionally,  we  were  told  that  of  all 
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fleet,"  were  the  depot's  most  important  customer.  Additionally,  we  were  told  that  of  all 
the  depot's  customers,  the  squadrons  were  probably  the  easiest  to  please,  largely  because 
they  least  understood  exactly  what  the  depot  does.  This  is  particularly  true  of  MCAPP, 
where  often  the  only  visible  evidence  of  work  performed  at  the  depot  is  the  aircraft's  new 
paint  job. 

F/A-18  program  personnel  at  the  depot  said  that  they  generally  did  not  receive 
enough  feedback  fi'om  squadrons  concerning  the  depot's  performance.  Direct  squadron 
interaction  with  the  depot  is  limited.  The  squadrons'  most  important  depot  point  of 
contact  is  usually  the  local  NADEP  field  team  leader.  This  individual  is  the  squadrons' 
point  of  contact  for  ISR  work,  drive-in  modifications,  and  PACE  inspections.  Once  an 
aircraft  has  been  sent  to  the  depot  for  MCAPP,  squadrons  often  obtain  status  information 
second  hand  fi'om  the  type  wing  staff,  although  NADEP  encourages  squadrons  to  call 
them  or  access  schedule  information  directly  on  the  depot's  electronic  bulletin  board 
system.  Squadrons  play  no  role  in  paying  or  accounting  for  depot  maintenance  fiands. 

The  next  level  of  NADEP  NI  customers  is  the  type  wing  staffs.  On  the  west  coast, 
the  Navy  F/A-18  type  wing  is  Commander,  Strike  Fighter  Wing  Pacific  (CSFWP)  based  at 
NAS  Lemoore,  CA.  On  the  east  coast,  the  Navy  F/A-18  t5q)e  wing  is  Commander,  Strike 
Fighter  Wing  Atlantic  (CSFWL)  based  at  NAS  Cecil  Field,  FL.  Marine  Corps  squadrons 
are  grouped  in  composite  wings,  like  Navy  carrier  air  wings,  with  multiple  aircraft  types  in 
each  wing.  Marine  Corps  F/A-18s  on  the  west  coast  belong  to  the  Third  Marine  Air 
Wing,  based  at  MCAS  Miramar,  CA.  Depot  coordination  functions  for  the  wing  are 
performed  by  the  attached  Marine  Aviation  Logistics  Squadron  1 1  (MALS  1 1).  Marine 
Corps  F/A-18S  on  the  east  coast  belong  to  the  Second  Marine  Air  Wing,  based  at  MCAS 
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Beaufort,  SC.  Depot  coordination  functions  for  the  wing  are  performed  by  the  attached 
Marine  Aviation  Logistics  Squadron  3 1  (MALS  3 1). 

The  Navy  type  wing  staflFs  and  Marine  Corps  MALS  maintenance  officers  are 
responsible  for  ensuring  that  their  squadrons  have  sufficient  aircraft  in  proper  material 
condition  to  meet  their  trmning  and  operational  commitments.  NADEP  serves  them  by 
producing  MCAPP  aircraft  on  schedule,  so  the  wing  can  deploy  fully  equipped  squadrons 
without  having  to  transfer  aircraft  among  squadrons.  The  wings  are  heavily  involved  in 
coordinating  depot  level  modification  programs  for  their  squadrons.  They  also  serve  as 
coordinators  for  the  depot's  on-site  drive-in  modification  programs.  However,  the  wings 
do  not  pay  for  depot  services.  Nevertheless,  they  are  aware  of  funding  issues  because  they 
attend  quarterly  Modification  Management  Meetings  and  interact  frequently  with  the 
depot  and  the  type  commander  staffs. 

The  next  level  of  NADEP  NI  customers  are  the  three  type  commander  staffs. 

They  are:  Commander,  Naval  Air  Forces  Pacific  Fleet  (CNAP)  in  San  Diego,  CA; 
Commander,  Naval  Air  Forces  Atlantic  Fleet  (CNAL)  in  Norfolk,  VA;  and  Commander, 
Naval  Air  Reserve  Force  (CNAR)  in  New  Orleans,  LA.  The  type  commander  staffs  are 
integrated  Navy  and  Marine  Corps  activities.  The  type  commander,  a  three  star  flag  billet, 
is  the  aircraft  controlhng  custodian.  With  only  a  few  small  exceptions,  all  Naval  and 
Marine  Corps  aircraft  belong  to  one  of  the  three  type  commanders  for  administrative 
purposes.  The  type  commander  staff  controls  and  directs  aircraft  transfers  between 
reporting  custodians  (squadrons).  Each  type  commander  staff  has  two  desks  that  interface 
with  the  depot  on  F/A-18  matters.  These  are  the  F/A-18  Class  Desk  and  the  Depot 
coordinator.  The  Class  Desk  is  the  focal  point  on  the  type  commander  staff  for  all  F/A-18 
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issues.  The  Depot  Coordinator  is  primarily  concerned  with  "big  picture"  aircraft  inventory 
management  and  funding  of  depot  maintenance  within  the  type  command.  The  Depot 
Coordinator  manages  the  type  commander's  depot  budget,  which  is  an  annual  allocation 
from  AIR  6.0.  Rework  needs,  priorities  and  funds  are  matched  within  the  type  command 
by  these  two  desks  working  together.  As  funding  for  depot  maintenance  is  very 
constrained,  the  type  commander  staffs  and  AIR  6.0  often  renegotiate  funding  allocations 
between  t5^e  commands  based  on  need.  The  type  commander  staffs  also  must  go  to  AIR 
6.0  for  over  and  above  funding  on  MCAPP  aircraft  that  require  more  than  10  percent 
additional  manhours  than  allowed  by  the  MCAPP  work  standard. 

The  final  level  of  NADEP  Mi's  customers  considered  in  this  thesis  is  the  Naval  Air 
Systems  Command  (NAVAIR)  headquarters.  NAVAIR  is  responsible  for  developing, 
procuring,  supplying,  and  supporting  all  aviation  systems,  related  equipment,  and  services 
required  by  the  Navy  and  Marine  Corps.  (NAVAIR  homepage)  The  specific  part  of 
NAVAIR  that  manages  aviation  depot  issues  is  AIR  6.0,  the  Assistant  Commander  for 
Industrial  Capabilities.  AIR  6.0  makes  policy  decisions  concerning  the  NADEPs, 
including  apportioning  workload  between  the  NADEPs,  other  government  depots  and 
commercial  repair  facilities.  As  stated  above,  AIR  6.0  allocates  funds  for  depot 
maintenance  to  the  type  commanders  and  controls  over  and  above  funding.  AIR  6.0  also 
works  closely  with  the  other  parts  of  NAVAIR,  such  as  AIR  3.0  (Logistics)  and  AIR  4.0 
(Engineering),  in  determining  depot  maintenance  requirements  and  the  best  ways  to 
perform  them.  As  a  depot  customer,  NAVAIR  is  concerned  primarily  with  cost  effective 
lifecycle  aircraft  support.  Due  to  declining  depot  maintenance  funds,  NAV AIR  is  always 
looking  for  ways  to  reduce  the  cost  of  depot  maintenance,  while  gaining  the  maximum 


24 


readiness  benefit  Ifroni  depot  dollars  spent.  Depot  turnaround  time  is  also  a  NAVAIR 
concern.  If  the  Navy  can  plan  on  a  shorter  turnaround  time  for  depot  maintenance,  they 
can  buy  fewer  aircraft  and  still  achieve  the  same  fleet  readiness. 

E.  CUSTOMER  SATISFACTION  MEASUREMENT  AT  NADEP  NI 

Customer  satisfaction  at  the  depot  is  measured  at  the  individual  program  level. 
Several  people  within  the  F/A-18  PMTO  at  NADEP  NI  are  measuring  and  tracking 
various  aspects  of  customer  satisfaction.  Overall  responsibility  is  informally  assigned  to 
the  program's  Quality  Competency  Manager.  Several  tools  are  in  place  to  measure  and 
track  F/A-18  program  quality  performance.  Some  of  these  are  direct  measures  of 
customer  satisfaction  and  others  are  related  indirectly  to  customer  satisfaction. 

The  direct  measures  of  customer  satisfaction  currently  used  by  NADEP  NI's  F/A- 
18  program  are  Aircraft  Discrepancy  Reports  (ADRs),  a  self-developed  squadron 
customer  survey  and  a  derived  customer  satisfaction  index  referred  to  as  the  GPA  (grade 
point  average). 

As  stated  previously,  squadrons  are  required  to  submit  an  ADR,  following  their 
acceptance  inspection,  on  aircraft  received  back  from  the  depot.  Reports  must  still  be 
sent,  even  if  there  are  no  discrepancies.  In  theory,  the  depot  should  receive  an  ADR  on 
every  aircraft  they  process  through  MCAPP  or  major  ISR  work;  in  some  cases  ADRs  are 
not  sent  by  the  squadron. 

The  ADR  uses  Standard  Form  368,  the  same  form  used  for  Category  2  Quality 
Deficiency  Reports.  As  a  result,  the  form  does  not  provide  optimum  customer  feedback 
on  the  depot.  Instructions  for  filling  out  the  ADR  are  located  in  OPNAVINST  4790. 2F. 
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the  depot.  Instructions  for  filling  out  the  ADR  are  located  in  OPNAVINST  4790.2F. 
Squadrons  may  only  list  legitimate  "discrepancies,"  classified  into  one  of  three  categories: 
Critical,  Major,  or  Minor.  A  copy  of  Standard  Form  368  is  shown  in  Figure  2-2.  The 
F/A-18  PMTO  responds  to  any  ADR  noting  a  discrepancy.  The  PMTO's  Quality 
Competency  Manager  immediately  contacts  the  squadron  by  telephone  to  inform  the 
squadron  that  NADEP  has  received  the  alleged  discrepancy  and  to  gain  additional 
information  Ifrom  the  squadron.  After  investigation  by  NADEP  NI  quality  assurance 
personnel,  the  ADR  discrepancy  is  either  confirmed  or  rejected,  and  the  squadron  is 
notified  in  writing,  with  a  copy  to  AIR  6.0.  If  the  discrepancy  is  confirmed,  any  corrective 
actions  taken  are  noted  in  the  response.  The  F/A-18  PMTO  tracks  the  number  of 
confirmed  ADR  discrepancies  and  publishes  this  information  in  a  series  of  three  charts. 
Examples  are  shown  in  Figures  2-3,  2-4  and  2-5.  These  CS  measures  are  some  of  several 
F/A-18  program  performance  charts  posted  in  a  highly  visible,  central  location  at  NADEP 
NI.  Copies  of  the  actual  ADR  reports  and  responses  are  posted  on  clipboards  in  work 
areas  for  the  depot's  artisans  and  supervisors  to  read. 

The  second  direct  CS  measure  currently  used  by  NADEP  NI,  a  "Quality  Process 
Improvement  Questionnaire,"  is  also  targeted  to  the  squadron  level  customer.  This 
questionnaire  is  provided  to  the  squadron  as  part  of  the  aircraft's  logbook  package,  on 
return  from  MCAPP  or  major  ISR  work  at  NADEP  NI.  The  questionnaire  is  divided  into 
three  sections.  The  first  section  asks  the  squadron  to  assign  a  numerical  grade  from  zero 
to  four  for  various  attributes  of  the  aircraft's  condition  on  return  from  the  depot.  The 
second  part  asks  the  squadron  to  assign  a  numerical  grade  using  the  same  scale  for 
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STANDARD  FORM  368  (REV.  10  85) 
GENERAL  SERVICES  ADMINISTRATION 
(FPMR  101-26.8) 


Figure  2-2.  Standard  Form  368  (front) 
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Figure  2-4.  F/A-18  PMTO  Confirmed  ADR  Defects  by  Type  Chart 
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different  attributes  of  the  depot's  customer  assistance/service.  The  third  section  asks  the 
squadron  to  write  in  any  general  comments  or  recommendations.  A  self  addressed 
stamped  envelope  is  provided  with  the  questionnaire  to  facilitate  its  return  to  the  depot. 
Despite  this,  not  all  questionnaires  are  returned.  The  questionnaire  also  has  a  block  for 
the  squadron  to  check  if  they  would  specifically  like  a  phone  response  from  the  depot  to 
any  of  their  comments.  One  problem  identified  with  the  questionnaire  concerns  who  in  the 
squadron  fills  it  out.  The  questionnaire  does  not  specify  or  recommend  a  particular 
respondant  within  the  squadron.  As  a  result,  data  received  is  inconsistent  and  may  come 
from  a  respondant  within  the  squadron  who's  not  cognizant  of  all  the  squadron's  quality 
concerns.  Copies  of  the  returned  questionnaires  are  also  posted  on  clipboards  in  work 
areas  for  the  depot's  artisans  and  supervisors  to  read.  A  copy  of  NADEP's  Quality  Process 
Improvement  Questionnaire  is  shown  in  Figure  2-6. 

A  third  direct  CS  measure  currently  used  by  NADEP  NI  is  a  composite  CS  index 
referred  to  as  the  GPA  (grade  point  average).  It  is  used  to  assess  CS  trends.  A  GPA 
score  is  obtained  from  each  Quality  Process  Improvement  Questionnaire  returned  to 
NADEP  NI.  The  GPA  score  is  the  arithmetic  average  of  the  scores  given  by  the  squadron 
to  the  eight  attributes  relating  to  the  aircraft's  condition  in  Section  A  of  the  questionnaire. 
Figure  2-7  is  a  copy  of  the  F/A-18  PMTO's  GPA  chart.  As  can  be  seen  in  Figure  2-7,  the 
scores  are  plotted  over  time,  using  NADEP's  aircraft  sequence  number.  A  simple  linear 
regression  line  is  also  included  in  the  chart. 

Several  indirect  measures  of  customer  satisfaction  are  currently  used  by  NADEP 
NI's  F/A-18  PMTO.  Schedule  and  internal  quality  are  the  focus  of  most  of  these 
measures.  Schedule  measures  focus  on  meeting  or  beating  the  110  day  turnaround  time 
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Quality  Process  Improvement  Questionnaire 

This  questionnaire  provides  NAVAL  AVIATION  DEPOT  NORTH  ISLAND  with 
invaluable  information  from  the  F/A-18  community.  The  feedback  you  provide  will 
be  used  for  our  continuous  process  improvements  efforts.  If  you  have  any 
questions  concerning  the  aircraft  you  have  received  or,  if  there  are  any  other 
services  we  can  provide  ,  please  contact  our  F/A-18  PMTO  office  at  DSN  735- 
4821  or  COMMERCIAL  (619)  545-4821,  FAX  #  735-3569.  LEGEND 

:  ■  Ni .  iSEQ.NO.:'-  . ;  -i; 


*BUNO.. 
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POOR- 

FAIR--^ 


AVERAGE-.- 
VERY  GOOD- 
EXCELLENT- 


A.  AIRCRAFT  CONDITION:  PLEASE  Em^ER  A  aiECK  MARK  IN  EAC3^ 
COLUMN  WHICH  BEST  REPRE  SENTS  EACH  CONDITION  ELEMENT, 


0 

1 

2 

3 

4 
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When  completed,  please  return  to: 

Commanding  Officer  Naval  Aviation  Depot,  Code  54200 

Naval  Air  Station,  Bldg  94 

P.O.  Box  357058  San  Diego,  Ca.  92135-7058 

(A  PRE-ADDRESSED  ENVELOPE  HAS  BEEN  PROVIDED) 


Figure  2-6.  F/A-18  PMTO  Quality  Process  Improvement  Questionnaire 
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Figure  2.7.  p/A-ig  pmto  GPA  Chart 


standard  set  for  MCAPP  at  NADEP  NI.  One  internal  quality  measure  tracks,  by  aircraft, 
the  number  of  work  orders  written  up  by  quality  assurance  personnel  (known  as 
Discrepancy  Work  Orders  or  DWOs)  while  inspecting  finished  tasks.  Another  internal 
quality  measure  tracks  the  number  of  required  maintenance  actions  written  up  and 
manhours  of  work  performed  by  NADEP  NX's  flight  line  maintenance  personnel  to  get  the 
aircraft  in  flyable  condition  after  MCAPP  or  major  ISR  work.  Like  the  direct  CS 
measures,  charts  displaying  NADEP's  performance  in  these  areas  are  posted  in  a  highly 
visible,  central  location. 

NADEP  NI  uses  customer  satisfaction  information  mostly  for  problem 
identification  and  process  improvement.  At  this  time,  there  is  no  formal  Imk  between  any 
customer  satisfaction  measures  and  employee  rewards  or  incentives.  Customer 
satisfaction  performance  is  factored  indirectly  into  some  performance  evaluations  and 
awards. 


F.  CUSTOMER  INTERFACING  AT  NADEP  NI 

Interviews  with  several  key  managers  within  NADEP's  F/A- 1 8  PMTO  stressed  the 
importance  they  place  on  talking  to  customers,  both  in  meetings  and  informally  by 
telephone.  They  saw  this  as  a  major  part  of  how  they  gauge  customer  satisfaction. 

F/A- 18  PMTO  managers  meet  face  to  face  with  wing,  type  commander  and  AIR  6.0 
personnel  at  least  quarterly  at  the  F/A- 18  Modification  Management  Meetings.  Face  to 
face  contact  with  the  squadrons  is  primarily  accomplished  by  NADEP's  on  site  field  team 
personnel  and  NADEP's  delivery  pilots  when  returning  an  aircraft  to  the  squadron.  The 
PMTO  staff  felt  that  it  was  easy  for  all  levels  of  customers  to  complain  ("Just  pick  up  the 
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phone").  They  also  believed  that  most  customer  concerns  were  received  and  handled  via 
telephone  conversations  with  various  members  of  the  PMTO  staff.  Though  customer 
concerns  voiced  formally  at  meetings  are  tracked  with  normal  action  item  procedures, 
there  is  no  system  within  the  PMTO  to  record,  track  and  follow  up  customer  issues 
identified  informally  or  via  telephone.  The  PMTO  does  not  provide  a  single  point  of 
contact  for  customers  to  call. 

NADEP  NI  provides  performance  and  schedule  information  to  customers  in 
several  ways.  Individual  requests  for  aircraft  status  or  other  information  are  handled  as 
received.  The  PMTO  briefs  MCAPP  schedule  performance  at  the  F/A-18  Modification 
Management  Meetings.  Actual  cost  and  schedule  data  for  completed  aircraft  are  entered 
into  NAV AIR'S  Naval  Aviation  Logistics  Data  Analysis  (NALDA)  database.  NALDA 
reports,  in  various  standard  and  custom  formats,  can  be  obtained  from  AIR  3.1.  In- 
process  aircraft  schedule  information  can  be  obtained  from  the  PMTO's  Logistics  Action 
Status  Tracking  (LAST)  system.  LAST  is  a  tool  designed  by  the  PMTO  to  provide  their 
customers  with  updated  schedule  information  concerning  any  F/A-18  aircraft  in  house  for 
either  MCAPP  or  in-service  repair.  LAST  is  a  computer  bulletin  board,  updated  by  the 
F/A-18  PMTO  once  every  two  weeks.  Interested  customers  can  obtain  passwords  by 
contacting  the  F/A-18  PMTO  at  NADEP  NI. 

G.  CONCLUSION 

NADEP  NI's  F/A-18  PMTO  is  commited  to  improving  their  performance  by  better 
understanding  and  measuring  customer  satisfaction.  This  is  not  a  simple  task.  The  PMTO 
has  several  different  levels  of  customers.  Not  all  customers  share  the  same  concerns,  and 
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the  customers  are  geographically  dispersed.  The  PMTO's  current  CS  measurement  system 
focuses  almost  entirely  on  the  squadron  level  customer  and  the  MCAPP/major  ISR 
product.  An  ideal  set  of  CS  measures  for  NADEP  NI's  F/A-1 8  PMTO  would  include  all 
external  customers  and  all  PMTO  products,  including  field  teams  and  engineering  support. 
For  such  a  tool  to  "pay  its  way,"  CS  measurement  results  should  be  integrated  into  PMTO 
decision  making  and  reward  systems. 


36 


m.  LITERATURE  REVIEW 


A.  INTRODUCTION 

The  purpose  of  our  literature  review  is  to  establish  a  thorough  understanding  of 
customer  satisfaction  before  attempting  to  develop  a  measurement  system  for  NADEP 
Ml.  Specifically,  this  literature  review  discusses  award  winning  customer  focused 
organizations,  establishes  the  need  to  measure  customer  satisfaction,  examines  current 
customer  satisfaction  measurement  methods,  presents  preconditions  to  measuring 
customer  satisfaction  and  sets  general  guidelines  for  measuring  customer  satisfaction. 

B.  CUSTOMER  FOCUSED  ORGANIZATIONS 

A  basic  tenet  of  total  quality  management  is  a  focus  on  the  customer  as  the  only 
significant  arbiter  of  the  quality  of  an  organization’s  output.  Accepting  this,  it  becomes 
critical  to  the  organization’s  success  to  determine  how  the  products  and  services  provided 
by  the  organization  are  valued  by  its  customers.  From  this  springs  the  science  of  customer 
satisfaction  measurement  (CSM).  The  theoretical  and  practical  importance  of  CSM  is 
widely  accepted  in  the  private/for  profit  sector,  as  illustrated  by  the  amount  of  time  and 
money  spent  by  businesses  on  CSM,  the  number  of  CSM  consultants  currently  offering 
their  services,  and  the  weight  allotted  to  CSM  programs  by  the  prestigious  Malcolm 
Baldridge  National  Quality  Award.  The  public/govemment  sector,  including  DoD,  is  also 
beginning  to  recognize  the  importance  of  CSM. 

A  1994  thesis  by  Casipit  and  Hodgkiss  documented  in-depth  the  CSM  practices  of 
several  quality  award  winning  organizations,  both  in  the  private  and  public  sectors.  Two 
of  the  public  sector  organizations  they  examined  are  similar  in  many  aspects  to  NADEP 
NI.  These  are  the  U.S.  Air  Force’s  Arnold  Engineering  Development  Center  (AEDC)  and 
Naval  Aviation  Depot,  Cherry  Point,  N.C.  Both  these  organizations  have  well  developed 
CSM  programs  integrated  "mth  their  operations. 
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AEDC  is  the  world’s  largest  aerospace  ground  test  facility.  AEDC  “conducts 
tests,  engineering  analyses,  and  technical  evaluations  for  research,  development  and 
operational  programs  of  the  Air  Force,  DOD,  other  government  agencies,  and  industry.” 
(Federal  Quality  Institute  1993)  Despite  DOD  downsizing,  AEDC  has  managed  to  grow 
as  a  result  of  its  TQM  based  philosophy.  AEDC’s  primary  objectives  are  all  directly 
related  to  customer  satisfaction: 

-  Achieve  100%  of  test  objectives  95%  of  the  time. 

-  Meet  test  start  dates  90%  of  the  time. 

-  Complete  95%  of  all  test  projects  at  or  below  estimated  cost. 

-  Reduce  the  average  customer  test  cost  by  10  %  over  the  next  two  years. 

-  Meet  customer  expectations  as  evidenced  by  an  average  rating  of  5.0  out  of  a 
possible  6.0  on  a  customer  survey.  (Federal  Quality  Institute  1993) 

AEDC’s  “objectives  have  built-in  performance  measures  or  metrics  which  are 
direct  indicators  of  mission  performance.”  (Hodgekiss  &  Casipit  1994)  These  goals  and 
metrics  were  all  developed  after  consultation  with  customers  to  determine  what  they 
considered  important.  These  goals,  widely  publicized  throughout  the  organization,  have 
the  added  benefit  of  reinforcing  customer-focused  behavior  by  all  employees.  (Hodgekiss 
&  Casipit  1994)  AEDC  measures  customer  satisfaction  both  indirectly,  by  monitoring  its 
performance  against  organizational  goals,  and  directly  by  means  of  a  customer  survey. 
Every  AEDC  customer  is  asked  to  complete  the  survey.  AEDC’s  customer  satisfaction 
survey  covers  five  areas:  planning,  financial  management,  program  management, 
schedule,  and  working  relationships.  Customers  rate  AEDC  on  a  six  point  satisfaction 
scale.  The  survey  has  proven  especially  useful,  as  it  tends  to  provide  more  actionable  data 
than  the  indirect  measures.  (Hodgekiss  Casipit  1994) 

Naval  Aviation  Depot,  Cherry  Point  (NADEP  CP)  is  one  of  three  surviving 
NADEPs.  There  were  six  NADEPs  prior  to  the  Base  Realignment  and  Closure  (BRAC) 
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process.  The  other  two  surviving  depots  are  NADEP  Jacksonville,  Florida,  and  the  subject 
of  this  thesis,  NADEP  NI.  NADEP  CP’s  mission  is  to  “provide  the  nation  with  the 
highest  quality,  worldwide  aviation  depot  level  maintenance,  engineering,  and  other 
logistics  support  on  time  and  at  the  least  cost.”  (Federal  Quality  Institute  1993)  All  of  the 
NADEPs  have  a  solid  foundation  in  TQM  as  a  result  of  the  strong  and  early  commitment 
of  their  parent  organization,  the  Naval  Air  Systems  Command.  Like  AEDC,  NADEP 
CP’s  CSM  program  also  employs  both  direct  and  indirect  methods  to  ensure  the  depot’s 
products  are  aligned  with  customer  expectations.  All  NADEPs  are  required  to  maintain  a 
Customer  Liaison  Program.  (OPNAVINST  4790.2F)  NADEP  CP  maintains  a  Customer 
Liaison  Office,  staffed  with  three  full-time  personnel;  this  is  NADEP  CP’s  single  face  to  its 
customers.  NADEP  CP  uses  direct  CSM  methods  such  as  monthly  telephone  surveys, 
annual  face-to-face  meetings  with  customers,  and  an  independent  survey  conducted  by 
Naval  Air  Systems  Command.  NADEP  CP  opted  for  telephone  surveys  instead  of  written 
surveys  or  customer  feedback  cards  after  they  experienced  common  problems  with 
written  survey  instruments;  a  low  response  rate,  no  assurance  that  the  “right”  people  were 
completing  the  surveys,  and  indications  that  responses  had  more  to  do  with  customer 
attitudes  than  customer  behavior.  NADEP  CP’s  indirect  methods  of  measuring  customer 
satisfaction  include  tracking  a  composite  quality  index,  and  cost  and  schedule 
performance. 

C.  WHAT  NEEDS  TO  BE  MEASURED? 

“The  first  step  in  measuring  customer  satisfaction  is  obviously  to  assess  the  quality 
and  performance  of  the  service  that  you  provide.”  (Urban  &  Wallace  1996)  As  an 
organization,  you  must  be  able  to  determine  how  your  product  or  service  is  fulfilling  the 
purpose  for  which  it  was  intended.  This  should  be  done  on  an  absolute  basis  and  as  a 
benchmark  relative  to  the  competition.  While  performance  measurements  are  key  to 
determining  customer  satisfaction.  Urban,  Wallace  &  Associates  (consulting  firm)  firmly 
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believe  there  are  three  additional  factors  that  provide  a  more  accurate  measure  of 
customer  satisfaction.  These  three  factors  are  meeting/exceeding  expectations,  quality  of 
the  interaction,  and  problem  resolution. 

Understanding  customer  expectations  and  then  meeting  or 
exceeding  them  is  fundamental  to  creating  satisfaction.  Customers  become 
satisfied  only  when  a  company  meets  or,  better  yet,  exceeds  their 
expectations.  This  is  true  regardless  of  the  intrinsic  quality  of  a  company’s 
product  or  service.  Thus  customer  expectations  are  as  important  as 
company  performance  in  determining  customer  satisfaction.  It’s  important 
to  recognize  as  well  that  expectations  are  not  static.  Performance  which 
satisfies  a  customer  today  may  not  be  sufficient  to  satisfy  the  same 
customer  in  tomorrow’s  competitive  environment.  As  a  result,  companies 
need  to  track  changes  in  customer  expectations  over  time  and  continually 
adjust  in  order  to  meet  those  changing  expectations.  (UrbanAVallace 
1996) 


The  quality  of  the  interaction  that  a  customer  has  with  an  organization  is  equally 
important  to  satisfying  the  customer.  “Quite  frequently,  how  a  customer  feels  about  the 
transaction  (how  he  or  she  is  being  treated)  is  actually  more  important  than  the  underlying 
quality  of  the  product  or  service  being  purchased.”  (UrbanAVallace  1996)  Customer 
satisfaction  metrics  should  measure  both  relationship  and  transactional  issues.  An  example 
of  a  relationship  issue  is  the  customer’s  intention  of  doing  business  with  an  organization 
again.  A  transactional  issue  would  be  one  that  deals  with  a  particular  interaction  or 
exchange  between  a  customer  and  an  organization.  A  customer  can  be  satisfied  with  the 
overall  performance  of  an  organization,  yet  dissatisfied  with  a  particular  transaction.  If 
customer  satisfaction  measures  are  only  set  up  to  measure  relationship  issues,  an 
organization  might  never  discover  dissatisfaction  with  particular  types  of  transactions. 

This  would  be  a  missed  opportunity  for  the  organization  to  get  data  to  improve  its 
processes. 
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Another  key  area  to  be  measured  is  customer  complaints.  This  is  one  of  the  most 
fruitfiil  areas  to  obtain  actionable  data  to  improve  an  organization’s  processes.  Quality 
award  winmng  firms  all  make  it  easy  for  the  customer  to  complain.  They  understand  that 
complaints  are  opportunities  to  exceed  customer  expectations  and  forge  even  stronger 
customer  loyalty.  (Gore  1996)  Techniques  used  include  1-800  phone  numbers,  24  hour 
service  lines,  customer  response  cards  provided  at  the  point  of  service,  and  customer 
service  representatives  who  may  travel  to  the  customer’s  location  to  assist  in  resolving 
problems.  (Gore  1996)  “In  many  cases,  we  have  found  that  good  problem  resolution 
actually  increases  customer  satisfaction  beyond  the  level  which  existed  before  the  problem 
occurred.”  (UrbanAVallace  1996) 

It  is  important  to  determine  what  factors  influence  customer  satisfaction.  Before 
customer  satisfaction  can  be  measured,  the  organization  “needs  to  know  the  criteria  used 
by  the  customers  to  evaluate  the  various  products  and  services.”  (OMB93)  There  are 
many  ways  to  do  this.  Some  research  advises  “listen[ing]  to  customers  and  front-line 
employees  to  obtain  their  definitions  of  the  agency’s  services  and  attributes  that  determine 
satisfaction  [and]  dissatisfaction.”  (OMB93)  In  the  case  of  a  NADEP,  broad  concerns 
shared  by  a  variety  of  customers  include  turnaround  time,  cost,  quality,  and  readiness  of 
the  delivered  product.  Metrics  for  these  factors  need  to  be  developed  that  are  accurate, 
simple,  understood  by  the  whole  organization,  provide  actionable  data,  and  that,  ideally, 
can  be  shared  with  the  customer. 

Another  set  of  customer  satisfaction  drivers  relevant  to  NADEP  NI  are  so-called 
“supplemental  factors.”  A  recent  study,  conducted  by  the  Sloan  Management  Review 
(1994),  found  an  important  distinction  that  has  developed  in  evaluating  services  is  the 
difference  between  one's  "core"  product  offering  and  one's  "supplemental"  (or  sometimes, 
"value-added")  services.  Examples  of  core  products  are;  safe  transport  from  one  city  to 
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another  via  airplane,  a  physician's  proper  diagnosis  and  treatment,  an  attorney's  sound 
legal  advice,  a  hotel  room  with  a  comfortable  bed  and  clean  bath,  the  car  to  be  purchased 
from  an  auto  dealer,  etc.  Examples  of  supplemental  products  are:  a  movie  and  meal  on 
board  the  airplane,  the  physician's  friendly  bedside  manner,  the  trustworthiness  of  the 
attorney,  bathroom  amenities  and  minibars  in  the  hotel  room,  and  the  car  dealership's 
financing. 


In  studies  of  customer  satisfaction  in  service  industries,  managers 
are  frequently  surprised  to  find  their  customers  are  judging  them  "on  the 
little  things"  (i.e.,  on  the  "supplemental").  There  are  good  reasons  for  this 
phenomenon.  First,  customers  assume  the  core  offering  will  be  of  high 
quality  —  it  is  a  given.  And  while  a  poor  "core"  will  result  in  customer 
dissatisfaction,  a  good  "core"  execution  is  not  sufficient  for  customer 
satisfaction.  A  supporting  reason  is  that,  within  and  across  competitors, 
there  is  typically  little  variability  in  the  core  product  offerings  --  planes 
usually  do  arrive  safely,  medical  treatment  is  fairly  accurate,  hotel  rooms 
usually  do  have  decent  bedding,  etc.  With  so  few  differences  among 
competitors  on  the  core  product  (or  within  a  competitor  across  different 
purchases),  this  information  is  not  distinctive  and  therefore  not  useful  to  a 
customer  forming  an  evaluation.  Furthermore,  most  consumers  find  the 
core  of  some  services  hard  to  judge  (e.g.,  most  do  not  have  the  expertise  to 
judge  an  attorney's  contracts  and  suggestions).  What  varies  more,  and  is 
easier  to  evaluate,  are  the  supplemental.  Interpersonal  skills  differ  greatly 
from  physician  to  physician  and  attorney  to  attorney,  and  hotel  room  and 
lobby  accouterments  also  vary  widely;  all  these  things  are  easy  to  judge. 
Thus,  in  an  evaluation  of  a  service  experience  or  in  a  choice  between 
service  providers,  supplemental  services  provide  greater  information  to 
consumers  and  become  those  features  of  the  product  offering  that  drive 
satisfaction  and  choice.  (lacobucci,  Grrayson,  and  Ostrom  1994) 


This  difficulty  in  judging  the  core  product  is  particularly  relevant  to  NADEP  NI.  Much  of 
the  work  that  NADEP  performs  on  an  aircraft  is  not  readily  apparent  or  visible  to  the 
customer.  Customers  may  form  much  of  their  opinions  of  depot  performance  on  more 
visible  supplemental  aspects  such  as  the  attitude  of  ferry  flight  crews  or  the  appearance  of 
the  aircraft’s  cleanliness  or  paint  job. 
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If  specific  performance  criteria  are  spelled  out  in  any  service  level  agreements 
(SLAs),  performance  specifications,  or  contracts  between  the  organization  and  a 
customer,  these  obviously  must  be  measured,  tracked,  and  reported  to  the  customer. 

SLAs  are  a  good  way  to  shape  customer  expectations  and  develop  closer  relationships 
between  organizations  and  their  customers.  (White96) 

D.  HOW  IS  CUSTOMER  SATISFACTION  MEASURED? 

Customer  satisfaction  can  be  measured  in  many  different  ways.  Many  current  top 
managers  take  that  statement  a  step  further  and  say  that  customer’s  satisfaction  must  be 
measured  in  different  ways  if  you  truly  want  to  get  an  accurate  picture  of  your  customer 
satisfaction.  Though  firms  often  express  a  desire  for  a  single,  simple  measure  of  customer 
satisfaction,  the  reality  is  that  a  variety  of  measures  is  generally  required  to  capture  the 
dynamics  of  customer  satisfaction.  A  1995  study  conducted  by  the  American  Marketing 
Association  surveyed  74  companies  about  their  customer  satisfaction  programs.  The 
research  found  that  the  predominant  methodologies  used  to  obtain  customer  satisfaction 
information  are  telephone  and  mail  surveys  (73%  and  62%,  respectively),  followed  by 
personal  interviews  (48%)  and  customer  focus  groups  (47%).  (Mentzer,  Bienstock,  and 
Kahn  1995)  The  survey  also  found  the  most  popular  types  of  analyses  are  cross 
tabulations,  qualitative  assessments,  descriptive  statistics  (e.g.,  means,  standard  deviations, 
percentages),  and  regression  analysis.  “Measures  can  be  classified  into  hard  measures 
(e.g.  financial  data,  production  data,)  perception  (interviews,  customer  surveys),  or  a 
combination  of  the  two,  called  Key  Process  Indicators.”  (Markowitz  96)  Regardless,  all 
metrics  chosen  by  an  organization  must  have  a  clearly  defined  purpose.  Collecting 
customer  satisfaction  data  which  doesn't  provide  actionable  data  or  isn't  somehow  used  to 
make  decisions  or  improve  processes  is  a  waste  of  time. 
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E.  SURVEYS 


The  survey  is  not  just  the  most  common  form  of  measuring  customer  satisfaction, 
it  tends  also  to  be  the  least  expensive.  However,  a  good  survey  is  not  easy  to  design  and 
takes  a  great  understanding  of  what  is  to  be  measured. 

Regardless  of  whether  the  people  developing  the  survey  are  internal 
or  external,  make  sure  they  understand  the  theoretical  basis  (e.g.,  gap, 
attitude,  value-attribute,  etc.)  of  the  instruments  and  are  familiar  with 
standard  procedures  for  developing  and  testing  reliable,  valid  items.  Keep 
in  mind  that  customer  satisfaction  survey  results  that  simply  describe  what 
was  found  provide  no  guidance  for  developing  an  action  plan  to  improve 
satisfaction.  (Mentzer,  Bienstock,  and  Kahn  1995) 

Although  every  organization  has  different  requirements  for  conducting  a  customer 
satisfaction  survey,  there  are  some  key  areas  that  must  be  considered  before  any  survey  is 
conducted.  The  Office  of  Management  and  Budget  (0MB)  published  a  step-by-step  guide 
for  public  sector  organizations  to  mount  a  customer  satisfaction  survey.  The  guide 
focuses  on  the  methodology  of  designing,  conducting,  and  analyzing  a  survey.  The 
following  12  steps  are  the  methodology  presented  in  the  0MB  guide: 

1.  Determining  the  scope  of  the  survey  measurement  program 

2.  Identifying  the  factors  and  characteristics  that  underlie  customer 
satisfaction 

3.  Identifying  the  target  customer  population  for  the  survey 

4.  Developing  a  sampling  frame  of  the  target  customers 

5.  Choosing  a  data  collection  method  best  suited  to  your  customer 
survey 

6.  Choosing  who  will  collect  the  survey  data  from  customers 

7.  Developing  and  pretesting  the  survey  questionnaire 

8.  Constructing  the  statistical  design  of  the  sample  of  customers 

9.  Designing  procedures  to  achieve  high  response  rates  in  the 
customer  survey 

10.  Ensuring  quality  while  the  survey  data  are  being  collected 

1 1 .  Processing  the  (returned)  surveys  and  preparing  them  for  analysis 

12.  Analyzing  the  data,  summarizing  the  results,  and  presenting  the 
findings  (0MB  93) 

Before  beginning  these  steps  there  are  certain  basic  planning  questions  that  must 
be  answered.  “What  are  your  agency’s  products/services?”,  “Who  are  the  customers  of 


44 


your  agency?”  and  “What  are  the  goals  of  your  survey?”  (0MB  93)  These  questions  may 
sound  simplistic,  however  they  need  to  be  answered  prior  to  planning  a  customer 
satisfaction  survey.  Surveys  must  be  constantly  reviewed  and  updated  if  necessary.  This 
seems  to  be  a  particular  problem  with  managers.  Dr.  Sionade  Robinson  and  Lyn 
Etherington  of  Cape  Consulting  in  a  recent  interview  stated  the  following  observation: 

Management  teams  believe  it  is  better  to  sacrifice  the  accuracy  of  customer 
satisfaction  measurements,  to  their  continuity.  In  other  words,  despite 
secretly  fearing  customer  satisfaction  surveys  may  no  longer  be  tracking 
the  right  issues,  they  prefer  to  persist  with  out-of-date  tools  in  order  to 
“track  our  performance  over  time.”  This  is  particularly  true  in 
organizations  who  have  linked  components  of  their  reward  and  recognition 
policies  to  customer  satisfaction  measurements.  (Robinson  &  Etherington 
1996) 

There  are  some  questions  that  the  customer  satisfaction  survey  must  answer.  If 
anything  changes  that  would  cause  the  survey  to  no  longer  answer  these  questions,  then  it 
must  be  revised.  These  questions  are: 

1 .  What  levels  of  service  do  customers  really  expect  from  a  supplier? 

2.  What  attributes  of  the  service  are  really  important  to  retaining  customers? 

3.  What  levels  of  service  do  customers  consider  unacceptable? 

A  recent  article  by  Mike  Donovan,  a  management  consultant,  stated  ten 
recommendations  for  measuring  customer  satisfaction  that  have  proved  to  be  noteworthy. 
The  list  is  a  good  summarization  of  key  points  that  an  organization  must  keep  in  mind 
when  measuring  customer  satisfaction. 

1 .  Do  not  delegate  customer  satisfaction  downward.  Take  an  active 
role  if  you  want  to  become  the  competitor  that  delights  customers. 

2.  Recognize  that  quality  is  a  given  and  that  response  time,  delivery 
reliability,  cost  and  value-added  services  will  be  required  to  gain  a 
competitive  advantage. 

3.  Understand  all  of  the  underlying  issues  that  prevent  your  company 
from  delivering  topnotch  customer  satisfaction. 
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4.  Resist  the  temptation  to  take  a  piecemeal  approach  to  customer 
satisfaction  improvement  simply  because  the  root  causes  of 
problems  seem  too  complex  and  interconnected  to  allow  an 
integrated  solution. 

5.  Base  customer  satisfaction  improvement  on  a  strong  executive 
directive  and  an  action  plan  containing  the  principles  and  tactics 
that  will  guide  the  organization  to  positive  and  permanent  change. 

6.  Survey  customers  to  find  out  what  they  think  your  company’s 
strengths  and  weaknesses  are  versus  your  competitors’.  “Listen” 
and  respond. 

7.  Focus  the  company’s  internal  activities  on  quality  and  response 
time  improvement. 

8.  Tie  the  measurement  system  for  customer  satisfaction  improvement 
to  the  reward  system  for  management  and,  if  possible,  all 
employees. 

9.  Conduct  regular  cross  fimctional  meetings  to  discuss  what’s 
working,  what’s  not,  and  what  actions  need  to  be  taken. 

10.  Develop  a  results-driven,  tactically  oriented  action  plan  with  the 
goal  of  providing  the  best  customer  satisfaction  in  your  industry. 
(Donovan  1996) 


F.  THE  “GAPS”  MODELS 

“Overall  satisfaction  is  affected  by  customer  expectations,  their  ideal  or  required 
performance,  and  actual  performance  of  the  agency.”  (OMB93)  A  significant 
development  influencing  the  study  of  service  quality  is  the  "gaps  model"  formulated  by 
Parasuraman,  Zeithaml,  and  Berry  in  1985.  While  some  experts  may  argue  its  degree  of 
effectiveness,  most  all  agree  that  it  should  be  used  to  begin  any  customer  satisfaction 
measurement  program.  This  model  is  grounded  in  disconfirmation  theory,  which  is  also  a 
prevalent  approach  to  studying  customer  satisfaction. 

Disconfirmation  theory  as  applied  in  service  quality  suggests  that,  before 
using  a  service,  a  client  has  certain  expectations  about  it.  After  the  service 
encounter,  he  compares  those  expectations  with  actual  performance  and  his 
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perception  is  either  confirmed  (if  they  match),  negatively  disconfirmed  (if 
the  perception  is  lower  than  the  expectation),  or  positively  disconfirmed  (if 
the  perception  is  higher  than  expectations).  The  essence  of  the  theory  is  a 
comparison  between  expectations  and  performance.  (Parasuraman  1994) 

The  gaps  model  focuses  on  several  service  gaps  that  affect  service  quality: 
between  customers’  and  management's  perception  of  service  expectations  (Gap  1); 
between  management's  perception  of  customers'  expectations  and  service-quality 
specifications  (Gap  2);  between  service-quality  specifications  and  actual  service  delivery 
(Gap  3);  and  between  actual  service  delivery  and  what  is  communicated  to  customers 
about  it  (Gap  4).  The  quality  gap  (Gap  5)  can  be  closed  by  reducing  the  four  internal  gaps 
found  within  the  management  of  a  service  organization.  In  measuring  service  quality  and 
applying  this  model,  however,  the  emphasis  has  been  on  the  "expected  service-perceived 
service  gap"  (Gap  5)  (P-E)  (Parasuraman  1994). 
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THE  GAPS  MODEL  OF  SERVICE  QUALITY 

(Parasuraman,  ZeithamI  and  Berry  1985) 
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Management 
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The  Organization 


In  1988,  to  test  the  gaps  model,  Parasuraman,  ZeithamI,  and  Berry  devised  the 
SERVQUAL  instrument  for  measuring  service  quality.  They  revised  it  slightly  in  1991. 
Since  the  gaps  model  was  derived  from  studies  in  several  different  service  industries,  the 
authors  intentionally  designed  a  "generic  instrument  with  good  reliability  and  validity  and 
broad  applicability."  (Parasuraman  1994)  No  other  instrument  for  measuring  service 
quality  has  been  tested  as  stringently  and  comprehensively  as  SERVQUAL. 

In  SERVQUAL,  the  client  responds  to  the  same  22  questions 
twice:  first,  to  establish  his  expectations  of  the  ideal  service;  then,  to  note 
his  perceptions  of  the  actual  service  provided  by  a  particular  firm.  Each 
response  is  scored  on  a  seven-point  Likert  scale.  Difference  scores  are 
computed  by  subtracting  the  score  for  expectations  from  the  perceptions, 
so  scores  can  range  from  -6  to  +6.  The  higher  the  score,  the  higher  the 
perception  of  quality.  (Parasuraman  1994) 
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The  SERVQUAL  mixed-model,  proposed  by  Parasuraman,  Zeithaml,  and  Berry 
( 1 994),  adds  a  third  survey  that  rates  the  importance  of  each  attribute.  The  formula  for 
measuring  the  service  quality  for  each  attribute  is: 

P  =  Perceived  performance  on  the  attribute  (i.e.,  the  amount  of  the 
attribute  possessed  by  the  object). 

E  =  Expectancy  norm. 

I  =  Importance  of  attribute 

SQ  =  Mixed-model  perceived  quality 

(1)  SQ  =  [(P  -  E)  D[l]  -  (P  -  E)  D[2]  +  ((I  -  E)  -  (P  - 1))  D[3]],  where 

D[l]  =  1  if  the  attribute  is  a  measurable  attribute,  or  if  the  perceived 
performance  is  less  than  or  equal  to  its  importance(P  </=  I)(true  in  most 
cases);  otherwise  D[l]  =  0. 

D[2]  =  1  if  the  attribute’s  expectancy  norm  is  equal  to  its  importance  rating 
(E=I)  and  the  perceived  performance  is  greater  than  its  importance  (P  >  I); 
otherwise  D[2]  =  0. 

D[3]  =  1  If  the  attribute's  expectancy  norm  is  less  than  its  importance  rating 
and  its  perceived  performance  is  greater  than  its  importance  (i.e.,  E  <  I  and 
P  >  I);  otherwise  D[3]  =  0.  (Journal  Of  Marketing  1996) 


The  following  is  an  example  of  a  survey  that  would  be  used  to  obtain  the  P,  E,  and  I 
factors.  This  example,  from  the  Journal  of  Marketing,  is  for  a  banking  institution: 

Expectations 

This  survey  deals  with  your  opinions  of  services.  Please  show  the 
extent  to  which  you  think  institutions  offering  banking  services  should 
possess  the  features  described  in  each  statement.  Do  this  by  using  the  scale 
presented  below.  If  you  strongly  agree  that  these  institutions  should 
possess  a  feature,  place  a  seven  on  the  line  preceding  the  statement.  If  you 
strongly  disagree  that  these  institutions  should  possess  a  feature,  place  a 
one  on  the  line.  If  your  feelings  are  not  strong,  place  one  of  the  numbers 
between  one  and  seven  on  the  line  to  properly  reflect  the  actual  strength  of 
your  feelings.  There  are  no  right  or  wrong  answers  —  all  we  are  interested 
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in  is  a  number  that  best  shows  your  expectations  about  institutions  offering 
banking  services. 


STRONGLY  1  2  3  4  5  6  7  STRONGLY 
DISAGREE  AGREE 

El .  They  should  have  up-to-date  equipment  &  technology. 

E2.  Their  physical  facilities  should  be  visually  appealing. 

E3.  Their  employees  should  be  well  dressed  and  appear  neat. 

E4.  The  appearance  of  the  physical  facilities  of  these  nstitutions 

should  be  in  keeping  with  the  type  of  services  provided. 

E5.  When  these  institutions  promise  to  do  something  by  a 

certain  time,  they  should  do  so. 

E6.  When  customers  have  problems,  these  institutions  shouldbe 

sympathetic  and  reassuring. 

E7.  These  institutions  should  be  dependable. 

E8.  They  should  provide  their  services  at  the  time  they  promise 

to  do  so. 

E9.  They  should  keep  their  records  accurately. 

ElO.  They  shouldn't  be  expected  to  tell  their  customers  exactly 

when  services  will  be  performed. 

Ell.  It  is  not  realistic  for  customers  to  expect  prompt  service 

from  employees  of  these  institutions. 

El 2.  Their  employees  don't  always  have  to  be  willing  to  help 

customers. 

E13.  It  is  okay  if  they  are  too  busy  to  respond  to  customer 

requests  promptly. 

E14.  Customer  should  be  able  to  trust  employees  of  these 

institutions. 

El 5.  Customers  should  be  able  to  feel  safe  in  their  transactions 

with  these  institutions'  employees. 

El 6.  Their  employees  should  be  polite. 

El 7.  Their  employees  should  get  adequate  support  from  these 

institutions  to  do  their  jobs  well. 

El 8.  These  institutions  should  not  be  expected  to  give  customers 

individual  attention. 

El  9.  Employees  of  these  institutions  cannot  be  expected  to  give 

customers  personal  attention. 

E20.  It  is  unrealistic  to  expect  employees  to  know  what  the  eeds 

of  their  customers  are. 

E2 1 .  It  is  unrealistic  to  expect  these  institutions  to  have  their 

customers'  best  interests  at  heart. 
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E22,  They  shouldn't  be  expected  to  have  operating  hours 
convenient  to  all  their  customers. 


Performance 


The  folloAving  set  of  statements  relate  to  your  feelings  about  XYZ.  For 
each  statement,  please  show  the  extent  to  which  you  believe  XYZ  has  the 
feature  described  by  the  statement.  Once  again,  placing  a  seven  on  the  line 
means  you  strongly  agree  that  XYZ  has  that  feature,  and  a  one  means  you 
strongly  disagree.  You  may  use  any  of  the  numbers  in  the  middle  as  well 
to  show  how  strong  your  feelings  are.  There  are  no  right  or  wrong 
answers  —  all  we  are  interested  in  is  a  number  that  best  shows  your 
perceptions  about  XYZ  whether  you  use  their  service  or  not. 


STRONGLY  1  2  3  4  5  6  7  STRONGLY 

DISAGREE  AGREE 

P 1 .  XYZ  has  up-to-date  equipment. 

P2.  XYZ 's  physical  facilities  are  visually  appealing. 

P3 .  XYZ 's  employees  are  well  dressed  and  appear  neat. 

P4.  The  appearance  of  the  physical  facilities  of  XYZ  is  in 

keeping  with  the  type  of  service  provided. 

P5.  When  XYZ  promises  to  do  something  by  a  certain  time,  it 

does  so. 

P6.  When  you  have  problems,  XYZ  is  sympathetic  and 

reassuring. 

P7.  XYZ  is  dependable. 

P8.  XYZ  provides  its  services  at  the  time  it  promises  to  do  so. 

P9.  XYZ  keeps  its  records  accurately. 

PIO.  XYZ  does  not  tell  its  customers  exactly  when  services  will 

be  performed. 

P 1 1 .  You  do  not  receive  prompt  service  from  XYZ  employees. 

P 1 2 .  Employees  of  XYZ  are  not  always  'willing  to  help 

customers. 

P13.  Employees  of  XYZ  are  too  busy  to  respond  to  customer 

requests  promptly. 

P14.  You  can  trust  employees  of  XYZ. 

P 1 5 .  You  can  feel  safe  in  your  transactions  with  XYZ's 

employees. 

P 1 6.  Employees  of  XYZ  are  polite. 

PI  7.  Employees  get  adequate  support  from  XYZ  to  do  their  jobs 

well. 

PI 8.  XYZ  does  not  give  you  individual  attention. 
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P 1 9.  Employees  of  XYZ  do  not  give  you  personal  attention. 

P20.  Employees  of  XYZ  do  not  know  what  your  needs  are. 

P21 .  XYZ  does  not  have  your  best  interests  at  heart. 

P22.  XYZ  does  not  have  operating  hours  convenient  to  all  their 

customers. 


Importance 

The  following  set  of  statements  relate  to  your  feelings  about  the 
importance  of  each  feature  described  in  your  decision  to  purchase  services. 
A  seven  means  you  consider  the  feature  very  important  in  deciding  where 
to  purchase  banking  services,  a  one  means  it  is  very  unimportant.  You  may 
place  any  of  the  numbers  shown  on  the  scale  below  beside  each  feature  to 
indicate  its  importance  to  you.  There  are  no  right  or  wrong  answers  --  all 
we  are  interested  in  is  your  perception  of  how  important  each  feature  is  to 
you  in  your  decision  where  to  purchase  banking  services. 


VERY  1  2  3  4  5  6  7  VERY 

UNIMPORTANT  IMPORTANT 

1 1 .  Up-to-date  equipment. 

12.  Physical  facilities  that  are  visually  appealing. 

13 .  Employees  that  are  well  dressed  and  appear  neat. 

14.  Physical  facilities  that  appear  to  be  in  keeping  with  the  type  of 
service  provided. 

15.  When  something  is  promised  by  a  certain  time,  doing  it. 

16.  When  there  is  a  problem,  being  sympathetic  and  reassuring. 

17.  Dependability. 

18.  Providing  service  by  the  time  promised. 

19.  Accurate  record  keeping. 

1 1 0.  Telling  the  customer  exactly  when  the  service  will  be  performed. 

111.  Receiving  prompt  service. 

112.  Employees  who  are  always  willing  to  help  customers. 

113.  Employees  who  are  not  too  busy  to  respond  to  customer  request 
promptly. 

114.  Employees  who  are  trustworthy. 

115.  The  feeling  that  you  are  safe  when  conducting  transactions  with 
the  firm's  employees. 

116.  Employees  who  are  polite. 

117.  Adequate  support  from  the  firm  so  employees  can  do  their  job 
well. 

118.  Individual  attention. 

119.  Employees  who  give  you  personal  attention. 

120.  Employees  who  know  what  your  needs  are. 
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121 .  A  firm  which  has  your  best  interests  at  heart. 

122.  Convenient  operating  hours. 

Other  Measures 

The  following  set  of  statements  relate  to  your  feelings  about  XYZ. 

Please  respond  by  circling  the  number  which  best  reflects  your  own 

perceptions. 

(Future  Purchase  Behavior) 

In  the  next  year,  my  use  of  XYZ  will  be 

NOT  AT  ALL  1  2  3  4  5  6  7  VERY  FREQUENT 

(Overall  Quality) 

The  quality  of  XYZ's  services  is 

VERY  POOR  1  2  3  4  5  6  7  EXCELLENT 

(Satisfaction) 

My  feelings  towards  XYZ's  services  can  best  be  described  as 

VERY  UNSATISFIED  1  2  3  4  5  6  7  VERY  SATISFIED 

(Journal  Of  Marketing  1996) 

SERVQUAL  is  often  used  in  conjunction  with  other  questions  which  assess  overall 
service  quality  or  evidence  of  subsequent  action,  e.g.  recommending  the  service  to  a 
fiiend,  or  willingness  to  use  the  service  again. 

Several  criticisms  have  arisen  about  the  SERVQUAL  scale  as  a  result  of  its 
widespread  use  and  close  scrutiny  by  other  researchers.  Most  have  been  rebutted  or 
addressed  in  subsequent  articles  by  Parasuraman  and  his  colleagues.  The  criticisms  have 
focused  on:  the  scale's  theoretical  base,  the  comparison  norms  for  "expectations,"  the 
number  and  generic  nature  of  the  dimensions,  the  instrument's  length,  the  ease  of 
administration  and  analysis  of  data,  the  need  to  use  both  perceptions  and  expectations 
data,  the  validity  of  difference  scores  as  data,  and  the  basis  for  inferring  that  higher  scores 
always  indicate  higher  quality. 
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In  numerous  studies,  researchers  have  reworded  items,  substituted  or  inserted  new 
items,  and  removed  items  from  the  scale  to  make  it  more  appropriate  for  the  service 
industiy  being  studied.  Such  modifications  are  not  considered  criticisms  of  SERVQUAL 
since  modifications  were  anticipated  and  suggested  by  the  original  developers.  As 
Parasuraman  and  his  collaborators  note,  however,  criticisms  and  findings  questioning  the 
number  and  nature  of  the  dimensions  may  arise  from  modifying  the  scale  so  much  that  its 
integrity  is  undermined.  Researchers  have  rarely  disputed  the  validity  of  the  individual 
items  or  statements  used  in  the  revised  scale,  considering  them  well-supported  by  the  scale 
development  and  revision  procedures  and  through  use  in  subsequent  studies.  As  a  result, 
the  actual  SERVQUAL  items  serve  as  the  basis  for  other  instruments. 

Several  of  the  criticisms  of  SERVQUAL  can  be  remedied  without 
rejecting  the  perception  of  service  quality  as  a  gap  between  performance 
and  expectations  or  the  P-E  approach.  Brown  and  others,  for  example, 
tested  an  alternative  to  difference  scores.  Addressing  definitional  problems 
with  the  term  "expectations,"  Parasuraman  and  his  cohorts  have  since 
clarified  expectations  as  "normative,"  not  prescriptive.  The  expectations 
represent  the  qualities  an  excellent  service  organization  would  have,  not 
what  it  should  have.  Word  changes  in  the  1991  revision  establish  that 
orientation  more  clearly.  But  other  criticisms  of  SERVQUAL  are 
interrelated  and  originate  in  its  definition  of  service  quality  as  a 
performance/expectations  gap.  Once  this  theoretical  approach  is  accepted, 
and  assuming  the  validity  of  the  dimensions,  the  instrument  must  measure 
both  expectations  and  performance  through  a  range  of  items,  resulting  in  a 
long  instrument.  Various  researchers  have  discovered  that  performance 
scores  alone  have  a  greater  predictive  value  for  overall  assessments  of 
service  quality  and  thus  question  the  need  for  both  measures.  As  a  result, 
within  the  last  few  years,  several  authors  have  developed  measures  based 
on  performance  alone.  (Journal  Of  Marketing  1996) 


The  movement  to  a  performance-based  measure  is  not  strictly  a  pragmatic 
response  to  difficulties  with  the  SERVQUAL  instrument.  Proponents  of  the  performance- 
based  methods  contend  that  multi-attribute  attitude  theory,  especially  the  "adequacy- 
importance"  model,  is  more  appropriate  than  the  gaps  model  and  disconfirmation  theory  if 
the  intent  is  to  predict  actual  behavior  or  behavioral  intent.  The  basic  premise  of  multi- 
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attribute  attitude  theory  is  that  clients  form  attitudes  about  service  or  product  quality  on 
the  basis  of  service  or  product  attributes.  The  argument  for  this  theory  is  that  it  better 
explains  relationships  between  service  quality,  customer  satisfaction,  and  purchase  or  use 
intentions. 

SERVPERF  is  another  model  used  to  generate  a  performance-based  measure  of 
service  quality.  It  was  developed  by  Cronin  in  1992  in  a  study  of  four  service  sectors 
(banking,  pest-control,  dry  cleaning,  and  fast  food).  Operationally,  SERVPERF  in  its 
final  form  omits  the  expectation  items  section  of  SERVQUAL.  SERVPERF  consists  of 
the  22  items  questioning  customers'  perceptions  of  service,  worded  exactly  as  in 
SERVQUAL.  It  may  include  questions  to  assess  the  importance  of  the  items'  dimensions 
and  several  questions  about  overall  service  quality,  satisfaction,  and  purchase  intention. 

As  in  SERVQUAL,  the  questions  can  be  modified  and  additional  items  included. 
SERVPERF  is  shorter  and  does  not  require  the  use  of  difference  scores  for  analysis. 
(Journal  Of  Marketing  1994) 

After  reviewing  the  different  "gaps"  models  and  the  published  criticisms  of  them, 
we  adapted  the  SERQUAL  mixed  model  of  service  quality  as  a  starting  point  to  develop  a 
set  of  tailored  customer  satisfaction  measures  for  NADEP  Mi's  F/A-18  maintenance 
program.  The  SERVQUAL-type  instrument  we  developed  is  presented  in  Chapter  IV.  It 
enabled  us  to  determine  where  differences  exist  between  customer  expectations  and 
performance  perceptions,  and  to  rank  the  relative  importance  to  the  customer  of  those 
differences.  From  this  data,  metrics  and  systems  were  developed  for  the  depot  to  use  to 
measure  the  satisfaction  of  each  of  its  four  different  types  of  customers. 

G.  IMPLEMENTING  A  CUSTOMER  SATISFACTION  PROGRAM 

The  Customer  Satisfaction  Program  must  be  unique  to  the  organization  if  it  is 
going  to  be  successful.  Success  depends  greatly  on  meeting  the  different  needs  of  all  the 
organization’s  stakeholders.  However,  there  are  some  fundamental  steps  in  implementing 
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a  Customer  Satisfaction  Program  that  are  basic  to  all  programs.  These  steps  were  well 
defined  in  a  recent  article  in  the  American  Marketing  Association’s 
Management  periodical.  The  eight  step  process  is  as  follows: 

1 .  Institute  a  process  to  tap  management,  employees,  outside 
consultants,  and  industry  sources  for  input  on  dimensions  critical 
to  customer  satisfaction. 

2.  Use  this  feedback  to  develop  an  ongoing  program  of  customer 
focus  groups  and  personal  interviews  to  identify  critical  customer 
satisfaction  dimensions. 

3.  Work  with  a  professional  staff  to  develop  telephone  and/or  mail 
survey  instruments  that  reliably  and  validly  incorporate  the 
identified  dimensions. 

4.  Ensure  employees  understand  the  theoretical  basis  of  the 
instruments  and  are  familiar  with  standard  procedures  for 
developing  and  testing  reliable,  valid  items. 

5.  Conduct  the  surveys  and  reevaluate  their  reliability  and  validity. 

6.  From  these  data,  develop  a  customer  satisfaction  metric  that  not 
only  relates  the  level  of  satisfaction  of  your  customers,  but  also 
analyzes  the  importance  of  the  various  dimensions  of  that 
satisfaction. 

7.  Use  the  dimensional  information  to  develop  an  action  plan  for 
improving  each  dimension  and  communicating  these  improvements 
to  customers.  (Delivery  of  customer  satisfaction  is  not  a  reality  if 
the  customer  does  not  notice  it.) 

8.  Tie  the  performance  evaluation  and  compensation  of  each  mployee 
involved  in  the  action  plan  to  its  accomplishment.  (Mentzer, 

Bienstock,  and  Kahn  1995) 


The  cost  of  implementing  a  customer  satisfaction  program  will  vary  from 
organization  to  organization.  The  periodic  review  of  customer  satisfaction  measures 
should  include  a  cost/benefit  analysis  of  the  different  measures  (measure  only  things  worth 
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measuring).  However,  an  organization  must  consider  the  cost  of  not  having  the  data 
obtained  from  different  measures,  when  conducting  a  cost/benefit  analysis. 


H.  CUSTOMER  SATISFACTION  PITFALLS 

A  recent  article  by  lacobucci,  Grayson,  and  Ostrom  pointed  out  many 
misconceptions  about  customer  satisfaction.  They  admit  that  service  quality  and  customer 
satisfaction  are  important  to  service  organizations  because  a  customer's  evaluation  of  a 
purchase  determines  the  likelihood  of  repurchase  and,  ultimately,  the  bottom-line  of 
business  success.  However,  they  bring  to  light  the  dangers  of  over-simplifying  the  concept 
of  customer  satisfaction. 

Simple  ideas  are  often  those  that  ‘catch  on’  fastest,  and,  true  to 
form,  the  gap  concept  is  popular  in  industry  and  academia.  Books  on 
customer  service  invariably  feature  examples  of  service  providers  who 
made  extra  efforts  to  please  their  customers.  Furthermore,  it  is  currently  in 
vogue  for  managers  in  many  industries  to  make  statements  such  as,  ‘We 
don't  want  to  just  meet  our  customers'  expectations,  we  want  to  exceed 
them,’  or  ‘We  don't  want  to  simply  satisfy  our  customers  (by  meeting 
expectations),  we  want  to  'delight'  them  (or  'amaze'  them)  by  exceeding 
their  expectations.  ’  Despite  the  persuasiveness  of  mangers  striving  to 
‘exceed  their  customers'  expectations,’  this  point  of  view  has  its  limitations. 

The  strength  of  the  concept  --  its  simplicity  —  is  simultaneously  its 
weakness;  it  is  too  simple  to  provide  a  thorough  understanding  of  customer 
evaluations.  We  recognize  that  these  ideas  have  taken  the  industry  by  storm 
and,  indeed,  seem  so  well  accepted  that  they  are  beyond  questioning. 

However,  we  feel  compelled  to  discuss  the  shortcomings  in  order  to  put  a 
brake  on  the  current  unquestioned  use  of  the  ‘exceeding  expectations’ 
ideology.  (lacobucci,  Grayson,  and  Ostrom  1994) 

According  to  lacobucci,  Grayson,  and  Ostrom  discovering  what  customers  want 
and  then  delivering  it  is  simply  good  marketing.  Attention  to  customers  is  what 
distinguishes  marketers  from  engineers  and  operations  personnel,  etc.  “We  are  not  saying 
that  a  focus  on  customer  satisfaction  is  not  necessary  or  is  a  bad  thing.  Indeed,  that  would 
be  like  sa5dng  marketing  is  not  necessary  or  is  bad.  If  it  takes  a  new  buzzword  to  refresh 
an  attention  to  the  consumer,  so  be  it.”  (lacobucci,  Grayson,  and  Ostrom  1994)  The  goal 
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of  the  satisfied  customer,  like  good  marketing,  must  permeate  the  entire  service  delivery 
process,  from  planning  through  execution.  If  customer  satisfaction  data  is  used  only  for 
post-purchase  assessment,  then  it  is  no  more  advanced  than  a  sales  force  counting  its 
receipts.  It  is  what  you  do  with  your  customer  satisfaction  data  that  is  important. 

Knowing  how  you  stand  in  your  customers’  eyes  is  interesting,  but 
it  is  only  valuable  if  the  information  is  used  to  improve  performance.  This 
is  the  point  at  which  some  customer  satisfaction  programs  break  down.  If 
you  tell  customers  that  you  are  interested  in  their  satisfaction  and  input, 
they  expect  you  to  act  on  that  information.  If  you  are  not  prepared  to 
follow  through,  the  whole  process  can  backfire.  (Zabusky  1995) 

A  recent  study  by  Cape  Consulting  found  several  problems  with  the  manner  in 
which  some  organizations  are  using  customer  satisfaction  data.  “Many  employees  find 
themselves  enduring  a  gruesome  ‘death  by  a  thousand  graphs’  of  aggregated  customer 
satisfaction  measures  on  service  attributes  over  which  they  feel  themselves  to  have  little  or 
no  influence.”  (Robinson  &  Etherington  1996))  The  problem  with  most  organizations 
seems  to  be  that  the  front-line  service  provider  perceives  aspects  of  quantitative  feedback 
to  be  removed  from  the  real  service  issues  that  they  understand. 

Another  issue  is  trying  to  understand  the  formats  in  which  customer  satisfaction 
survey  results  are  presented.  What  does  scoring  a  5.6  on  cooperativeness  mean?  Is  it 
good  or  bad?  Without  clear  explanations  or  goals,  many  customer  satisfaction 
measurements  end  up  being  just  meaningless  information  presented  at  staff  meetings. 
Effective  customer  satisfaction  measures  give  a  clear  understanding  of  the  organization's 
present  position  in  a  given  attribute.  The  attributes  and  their  relationship  to  the 
organization's  goals  must,  in  turn,  be  understood  well  enough  so  that  strategies  for 
improvement  can  be  developed. 

Some  managers  are  running  around  proclaiming  "The  Customer  Is  King"  and 
sketching  organizational  charts  with  customers  positioned  where  CEOs  used  to  sit.  While 
achieving  customer  satisfaction  is  an  admirable  goal,  an  organization  must  answer  to 
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multiple  sets  of  "customers"  (e.g.,  consumers,  boards,  shareholders),  or  in  the  case  of 
NADEP  NI,  squadrons,  wings,  type  commands  and  NAVAIR,  whose  goals  may  be  in 
conflict.  For  example,  while  a  squadron  may  wish  for  an  aircraft  to  be  completely 
repainted  as  part  of  the  depot  process,  NAVAIR  may  not  consider  this  financially  feasible 
or  desirable  from  an  engineering  standpoint.  Likewise,  the  squadron  may  expect  the 
airplane  to  return  from  the  depot  in  a  Full  Mission  Capable  (FMC)  status,  while  the  type 
command  may  be  more  concerned  with  the  additional  turnaround  time  required  to  achieve 
FMC  status.  In  these  cases,  the  expectations  of  all  customers  must  be  examined.  So 
before  attempting  to  satisfy  all  customers,  an  organization  must  identify  all  customers’ 
expectations  and  rank  them  according  to  importance  in  achieving  the  organization’s 
mission. 

lacobucci,  Grayson,  and  Ostrom  claim  the  phrase,  "The  Customer  Is  Always 
Right,"  is  utter  nonsense.  They  point  out  studies  of  product  liability  that  consistently 
attribute  at  least  half  of  product  failures  to  consumer  misuse.  In  addition,  the  briefest 
inquiry  of  any  sales  force  will  confirm  that  some  customers  are  uninformed,  unrealistic, 
and  demanding.  Most  businesses  have  certain  segments  of  customers  who  are  not 
profitably  worth  satisfying.  This  raises  the  question  about  customer  education.  In  private 
industry,  competition  drives  consumers  to  seek  out  the  best  service  provider.  However,  in 
the  case  of  a  government  agency  or  sole  source  provider,  the  customer  has  little  or  no 
incentive  to  understand  the  capabilities  and  constraints  of  a  service  provider.  This  lays 
some  burden  on  the  organization  to  educate  the  customer,  in  order  for  them  to  have 
realistic  expectations  of  service. 

Customers  evaluate  purchases  as  an  aggregate  function  of  a  number 
of  factors.  Value,  or  the  tradeoff  between  the  quality  of  the  item  and  its 
costs,  is  a  primary  consideration.  In  essence,  this  judgment  is  one  of  equity 
—  how  do  the  outcomes  rate  (e.g.,  the  quality  of  what  I  receive)  relative  to 
the  inputs  (e.g.,  the  price  I  paid  or  efforts  and  costs  I  incurred).  Notice 
too  that  a  simple  derivation  of  the  desire  to  "exceed  expectations"  would 
be  to  sell  products  at  costs  low  enough  to  be  unprofitable  to  businesses. 
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e.g.,  giving  away  a  Mercedes  would  no  doubt  satisfy  (and  even  "delight")  a 
customer.  While  many  businesses  are  seeking  high  levels  of  customer 
satisfaction,  none  would  do  so  rationally  if  it  meant  jeopardizing  their  long¬ 
term  existence.  TLiC  "exceed  expectations"  perspective  would  be  more 
thoughtful  and  useful  if  such  constraints  were  also  explicitly  considered. 

(lacobucci,  Grayson,  and  Ostrom  1994) 

1.  CONCLUSION 

Our  literature  review  provided  us  with  the  thorough  understanding  of  customer 

satisfaction  necessary  to  develop  a  measurement  system  for  NADEP  NI.  The  following  is 

a  list  of  the  issues  most  relevant  to  producing  a  tailored  set  of  customer  satisfaction 

measures  for  NADEP  NI: 

1)  Identify  all  external  customers  and  consider  their  unique  requirements. 

2)  Government  agencies  must  “educate  their  customers”  to  ensure  their  customers 
have  appropriate  expectations.. 

3)  The  “gaps  model”  can  be  an  effective  tool  if  properly  fitted  to  the  organization. 

4)  Measures  must  be  linked  to  processes,  and  give  actionable  data 

5)  The  “right  things”  should  be  measured,  and  resources  should  not  be  wasted 
gathering  measurement  data  that  will  not  be  acted  on. 

6)  A  variety  of  measures  is  needed,  direct  and  indirect,  to  truly  understand  the 
satisfaction  level  of  an  organization's  customers. 

7)  It  is  critical  that  customer  satisfaction  measurement  data  be  integrated  into  an 
organization’s  decision  making  processes. 

8)  Customer  satisfaction  measurements  need  to  be  periodically  reevaluated  to  ensure 
that  the  organization  is  still  measuring  the  right  things. 
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IV.  PRETEST  QUESTIONNAIRE  DEVELOPMENT 


A.  INTRODUCTION 

In  order  to  develop  a  customer  satisfaction  measurement  system  that  is  truly  useful 
to  NADEP  NI,  we  had  to  understand  NADEP  NI’s  customer’s  expectations  and  their 
attitudes  about  performance  and  the  importance  of  different  attributes  of  F/A-18  depot 
maintenance.  We  developed  a  pretest  questionnaire  which  was  sent  to  several  of  NADEP 
NI’s  customers.  This  pretest  questionnaire  was  adapted  from  the  SERVQUAL  mixed 
model  (Parasuraman,  Zeithaml,  and  Berry  1994)  and  modified  according  to  our  data 
needs.  The  information  obtained  in  responses  to  this  pretest  questionnaire  is  paramount  in 
developing  our  tmlored  customer  satisfaction  measurement  system  for  NADEP  NI. 

B.  BACKGROUND 

The  background  information  used  to  develop  this  pretest  questionnaire  was 
obtained  through  interviews  with  squadron  maintenance  officers,  squadron  commanders, 
wing  maintenance  officers,  type  command  class  desk  and  depot  coordinator  personnel, 
and  NAVAIR  Code  6.0  personnel.  These  interviews,  were  conducted  both  telephonically 
and  in  person.  During  these  interviews  we  tried  to  determine  each  customer’s  opinions 
about  F/A-18  depot  maintenance.  We  tried  to  keep  the  interviews  as  open  ended  as 
possible,  allowing  the  customers  to  express  all  possible  concerns. 

C.  MODEL  IDENTIFICATION 

We  decided  to  adapt  the  SERVQUAL  mixed  model  (Parasuraman,  Zeithaml,  and 
Berry  1994)  because  we  felt  that  it  best  represented  the  type  of  analysis  required  to 
develop  a  customer  satisfaction  measurement  system  for  NADEP  NI.  This  model  has 
been  used  by  many  companies  to  accurately  measure  customer  satisfaction.  These  well 
docuemented  results  were  the  basis  of  our  decision  to  use  SERVQUAL.  We  felt  that 
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understanding  the  gaps  between  importance,  expectations  and  performance  of  the  different 
attributes  would  focus  our  research  in  developing  a  customer  satisfaction  measurement 
system.  This  model  also  captures  differences  in  customer  t5T3e  and  customer  location, 
when  responses  are  sorted  by  these  criteria. 

D.  ATTRIBUTE  IDENTIFICATION 

The  four  attributes  that  we  measured  are  quality,  turnaround  time,  cost  and 
customer  relations  with  the  depot.  These  attributes  were  suggested  by  our  interviews  with 
customers,  NADEP  M’s  current  customer  satisfaction  measurement  system,  and  our 
literature  review,  including  NAVAIR  Instruction  13023.1  (Naval  Aviation  Depot 
Workload  and  Material  Standards  Required  for  the  Aircraft  and  Engine  Programs). 

From  our  interviews,  it  was  clear  that  different  customers  considered  different  attributes 
more  important.  Our  pretest  questionnaire  was  designed  to  clearly  make  these  distinctions. 

The  attribute  of  quality,  for  the  purpose  of  the  pretest  questionnaire,  deals  directly 
with  the  finished  product,  delivered  aircraft  in  this  case.  We  developed  quality  related 
questions  to  cover  the  areas  that  have  the  greatest  impact  on  NADEP  M’s  F/A-18 
customers.  The  questions  on  turnaround  time  were  developed  to  capture  customers’ 
scheduling  concerns  as  well  as  the  operational  impact  of  F/A-18  depot  maintenance.  The 
cost  questions  assess  each  customer’s  knowledge  and  concern  about  the  cost  associated 
with  NADEP  M’s  F/A-18  maintenance  program.  The  relationship  questions  were 
developed  to  assess  the  “value  added”  attributes  of  NADEP  M’s  F/A-18  program.  While 
these  attributes  are  usually  the  easiest  to  improve,  our  literature  review  indicated  that  they 
often  have  the  greatest  impact  on  customer  satisfaction. 
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E.  QUESTIONNAIRE  FORMAT 


In  our  literature  review,  we  found  that  personal  interviews,  telephone  interviews 
and  self-administered  questionnaires  (mail  surveys)  are  the  most  common  methods  of 
collecting  data.  Figure  4-1  lists  the  advantages  and  disadvantages  of  each  method.  Due  to 
the  cost  and  time  constraints,  we  chose  to  use  a  self-administered  questionnaire  as  our 
prime  data  collection  method. 

A  copy  of  the  pretest  questionnaire  is  enclosed  at  the  end  of  this  chapter.  The 
questionnaire  is  divided  into  three  sections.  The  first  section  deals  with  the  customers’ 
general  expectations  concerning  F/A-18  depot  maintenance.  The  second  section  evaluates 
the  customers’  perceptions  ofNADEP  NI’s  F/A-18  maintenance  program  performance. 
The  final  section  gathers  customers’  assessments  about  the  importance  of  the  different 
aspects  of  F/A-18  depot  maintenance.  Each  section  of  the  pretest  questionnaire  is  divided 
into  5  parts.  The  first  four  parts  contain  23  statements  that  pertain  to  the  attributes  of 
quality,  turnaround  time,  cost  and  depot  relations.  The  fifth  part,  called  “other,”  deals 
with  specific  concerns  that  we  encountered  during  our  research.  At  the  end  of  each 
section,  there  is  space  for  the  respondent  to  make  additional  comments  about  the  section 
or  the  questionnaire  in  general. 

The  questionnaire  was  developed  with  the  understanding  that  all  respondents  will 
be  middle  and  upper  level  maintenance  managers  Avith  a  least  some  working  experience  at 
NADEP  NI.  In  developing  the  pretest  questionnaire,  we  felt  it  important  to  make  the 
questionnaire  as  easy  as  possible  to  fill  out  and  return.  We  therefore  used  the  following 
criteria  in  developing  our  pretest  questionnaire: 
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1.  Personal  interviews: 

Advantages: 

*  Use  a  more  representative  sample. 

*  Achieve  higher  return  rate. 

*  Produce  fewer  incomplete  questionnaires. 

*  More  questions  can  be  asked. 

*  More  complex  measurement  methods  can  be  used. 

*  Responses  may  be  more  readily  verfied. 

Disadvantages: 

*  More  costly 

*  Subject  to  interviewer  bias,  error  and  cheating. 

*  Subject  to  response  bias. 

2.  Telephone  interviews: 

Advantages: 

*  Can  be  conducted  quickly 

*  Relatively  low  cost. 

Disadvantages: 

*  Sample  bias. 

*  Usually  must  be  brief. 

3.  Self-Administered  questionnaires: 

Advantages: 

*  Least  costly. 

*  Avoid  interview  bias. 

*  Larger  number  of  respondents  can  be  reached. 

*  More  convenient  to  the  respondent. 

*  Requires  a  smaller  staff  for  administering. 

Disadvantages: 

*  Sample  is  almost  certainly  not  likely  to  be  “representative”  without  follow  up. 

*  Must  be  carefully  designed  and  pretested  to  avoid  respondent  confusion. 


Figure  4-1  Data  Collection  Methods  (Advantages  and  Disadvantages) 
(Morris  and  Bird  well  1988) 
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1)  Clear  and  concise  instructions. 

2)  Understandable  and  unambiguous  measurement  scales. 

3)  Easy  to  answer,  closed-ended  questions. 

4)  A  section  to  provide  additional  comments  about  customer  satisfaction  and 

the  questionnaire. 

5)  Minimize  the  number  of  questions  and  time  required  to  complete  the 
questionnaire,  while  still  providing  the  necessary  data. 

6)  Professional  appearance. 

7)  A  self  addressed  stamped  envelope  provided  with  each  questionnaire. 

F.  MEASUREMENT  INSTRUMENT 

There  are  a  multitude  of  different  measuring  instruments  to  collect  data.  During 
our  literature  review,  the  most  common  marketing  measurements  were  the  semantic 
differential  scale,  Likert  scales  and  paired  comparisons.  Figure  4-2  shows  examples  of 
these  measuring  instruments. 

We  decided  to  use  a  version  of  the  Likert  scale.  This  instrument  provides  data  in 
the  most  relevant  form.  Also,  the  SERVQUAL  model,  from  which  we  adapted  our 
pretest  questionnaire,  uses  the  Likert  scale.  We  decided  to  use  a  range  of  only  five 
choices,  instead  of  the  usual  seven  on  a  Likert  scale,  due  to  the  limited  population  and 
small  number  of  questioimaires  administered.  For  the  “Expectations”  and  “Performance” 
sections  of  the  pretest  questionnaire,  our  version  of  the  Likert  scale  used  the  following 
definitions: 

5=Agree  Strongly 
4= Agree 
3=Undecided 
2=Disagree 
l=Disagree  Strongly 
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The  “Importance”  section  of  the  pretest  questionnaire  used  the  following  Likert  scale: 

5= Very  Important 
4=Important 
3=Undecided 
2=Unimportant 
l=Very  Unimportant 

We  strongly  felt  that  the  customers  would  be  able  to  clearly  express  their  opinions  using 
this  modified  scale. 


MEASUREMENT  INSTRUMENTS 

SEMANTIC  DIFFERENTIAL  SCALE 

“Would  you  rate  General  Motors  as  being: 

Progressive 

Conservative 

12  3  4 

5  6  7 

Strong 

Weak?” 

12  3  4 

5  6  7 

LIKERT  SCALE 

PAIRED  COMPARISONS 

“Ajax  is  an  excellent  cleanser.” 

Do  you  prefer  Brand  A  or  Brand  B? 

Agree  extremely  strongly 

Agree  fairly  strongly 

Do  you  prefer  Brand  C  or  Brand  A? 

Agree 

Undecided 

Disagree 

Disagree  fairly  strongly 

Disagree  extremely  strongly 

Do  you  prefer  Brand  B  or  Brand  C? 

Figure  4-2  Commonly  Used  Types  of  Rating  Methods  and  Scales 
(Morris  &  Birdwell  1988) 
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G.  PROCEDURE 


The  step  by  step  procedure  used  to  administer  our  pretest  questionnaire  is  as 

follows: 

1 .  Gained  approval  from  NADEP  NI  to  administer  the  pretest  questionnaire. 

2.  Received  from  NADEP  NI  a  list  of  all  squadrons  that  had  received  an 
F/A-1 8  aircraft  back  from  NADEP  NI  (either  from  MCAPP  or  major  ISR  work)  in  the  last 
18  months. 

3.  Contacted  east  and  west  coast  USN  and  USMC  F/A-1 8  wing  maintenance 
officers  to  arrange  their  assistance  in  distributing  the  pretest  questionnaire  to  all  concerned 
squadrons. 

4.  Contacted  NAVAIR  and  the  type  command  staffs  to  alert  them  the  pretest 
questionnaire  was  enroute. 

5.  Mailed  all  pretest  questionnaires  with  self  addressed  stamped  envelopes. 
Squadron  questionnaires  were  sent  via  their  wing  maintenance  officers,  where  possible,  for 
distribution. 

6.  Called  customers  approximately  one  week  later  to  ensure  they  had  received 
the  pretest  questionnaire. 

H.  CONCLUSION 

The  information  gathered  while  developing  this  pretest  questionnaire  and  analyzing 
the  resulting  data  forms  the  basis  for  identifying  a  tailored  set  of  customer  satisfaction 
measures  for  NADEP  NI.  This  pretest  questionnaire  seems  to  have  captured  the  most 
important  factors  of  customer  satisfaction  for  the  NADEP  NI  customer.  Analyzing  the 
gaps  in  expectations,  performance  and  importance,  can  focus  NADEP  NI’s  efforts  in 
improving  customer  satisfaction. 
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NADEP  NORTH  ISLAND  F/A-18  PROGRAM 
CUSTOMER  SATISFACTION  PRETEST  QUESTIONNAIRE 

Dear  NADEP  North  Island  Customer, 

We  are  students  at  the  Naval  Postgraduate  School  in  the  Systems  Management 
Department  currently  developing  a  set  of  customer  satisfaction  measures  for  Naval 
Aviation  Depot  North  Island  (NI).  This  work  is  being  done,  at  the  request  of  NADEP 
NI,  for  our  thesis  requirement.  NADEP  NI  is  genuinely  concerned  with  providing  its 
customers  with  the  best  service  possible.  In  order  to  accomplish  this,  NADEP  NI  must 
have  a  thorough  understanding  of  the  perceptions,  expectations  and  importance  factors 
that  each  of  its  customers  have  towards  the  services  that  it  provides. 

This  questionnaire  meets  the  "pretest"  criteria  of  draft  OPNAV  Instruction 
5300. 8B.  It  will  assist  us  in  the  development  of  data  collection  instruments  that  will 
ensure  NADEP  NI  can  properly  measure  its  customers'  satisfaction  levels.  This 
questionnaire  was  developed  based  on  comments  from  NADEP  NI  customers  whom  we 
interviewed.  Please  assist  us  in  developing  these  important  measurement  instruments  by 
completing  the  enclosed  questionnaire.  Feel  free  to  write  any  comments  concerning 
specific  questionnaire  items  any  place  on  the  form.  At  the  end  of  each  section,  there  is  an 
area  for  you  to  write  comments  or  suggestions  that  you  think  would  improve  customer 
satisfaction. 

Please  return  the  completed  survey  in  the  accompanying  self-addressed  stamped 
envelope  provided,  or  mail  to: 

CDR  Brian  Forsyth,  USN 

Naval  Postgraduate  School,  SGC#  2357 

Monterey,  CA  93943 

We,  as  well  as  NADEP  NI,  greatly  appreciate  the  time  and  effort  you  spend  in 
completing  this  questionnaire.  We  believe  that  this  work  spent  to  develop  a  customer 
satisfaction  measurement  system  will  lead  to  your  improved  satisfaction  as  a  NADEP  NI 
customer.  Thank  you  for  your  interest  and  cooperation. 

Brian  A.  Forsyth,  CDR,  USN 


John  P.  Chadboume,  CPT,  USA 


NADEP  NORTH  ISLAND  F/A-18  PROGRAM 
CUSTOMER  SATISFACTION  PRETEST  QUESTIONNAIRE 

Please  fill  in:  your  activity's  name 

your  billet  title 

PART  ONE:  EXPECTATIONS 

This  portion  of  the  survey  deals  with  your  opinions  of  F/A-18  aircraft  depot  maintenance  services  in 
general,  whether  performed  at  NADEP  North  Island  or  anywhere  else.  Please  show  the  extent  to  which 
you  think  depots  offering  F/A-18  aircraft  maintenance  services  should  possess  the  characteristics 
described  in  each  statement.  Do  this  by  using  the  scale  presented  below.  If  you  strongly  agree  that  depots 
should  possess  that  characteristic,  circle  the  number  five  on  the  same  line.  If  you  strongly  disagree  that 
they  should  possess  that  characteristic,  circle  the  number  one  on  the  same  line.  If  your  feelings  are  not 
strong,  circle  the  number  between  one  and  five  that  best  reflects  the  strength  of  your  feelings.  There  are 
no  right  or  wrong  answers  —  all  we  are  interested  in  is  a  number  that  best  shows  your  expectations  about 
depots  offering  F/A-18  aircraft  maintenance  services. 

l=Disagree  Strongly  2=Disagree  3=Undecided  4=Agree 

5=Agree  Strongly 

QUALITY 

Aircraft  delivered  to  squadrons  after  MCAPP  should  be  FMC 

1  2  3 

4 

5 

Aircraft  delivered  from  depot  should  be  free  of  quality  defects 

1  2  3 

4 

5 

Aircraft  delivered  from  MCAPP  should  have  a  “good”  paint  job 

1  2  3 

4 

5 

Aircraft  logbooks  and  records  delivered  from  depot  should  be 
discrepancy  free 

1  2  3 

4 

5 

Aircraft  delivered  from  depot  should  be  clean 

1  2  3 

4 

5 

Aircraft  delivered  from  depot  should  be  FOD-firee 

1  2  3 

4 

5 

PACE  inspection  criteria  should  be  consistent 

1  2  3 

4 

5 

All  modification  or  special  rework  items  requested  by  the  squadron 
should  be  accomplished  during  MCAPP 

1  2  3 

4 

5 

TURNAROUND  TIME 

Depot  maintenance  should  not  impact  operational  requirements 

1  2  3 

4 

5 

Aircraft  should  complete  depot  maintenance  on  schedule 

1  2  3 

4 

5 

Customers  should  be  notified  immediately  of  changes  to  scheduled 
completion  dates 

1  2  3 

4 

5 
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COST 


My  activity  should  be  concerned  about  aircraft  depot  maintenance  costs  1 

Aircraft  MCAPP  cost  should  be  related  to  actual  depot  man-hours  expended  1 


RELATTONSHTP 

My  depot  point-of-contact  should  be  easy  to  get  a  hold  of  1 

The  OPNAV,  NAVALR.  and  local  procedures  for  doing  business  1 

with  tlie  depot  should  be  designed  for  easy  interaction  with  the  depot 

The  depot  should  respond  in  a  timely  manner  to  requests  for  services  1 

The  depot  should  respond  in  a  timely  manner  to  customer  complaints  1 

The  depot  should  inform  customers  about  the  specific  work  1 

performed  on  their  aircraft  as  part  of  MCAPP 

Aircraft  schedule  status  information  should  be  easy  to  obtain  1 

The  depot  should  visit  customers  periodically  to  assess  their  needs  1 

The  depot  should  provide  a  point  of  contact  to  facilitate  customer  1 

complaints/requests 

Depot  employees  should  deal  with  customers  in  a  courteous  and  1 

helpful  matmer 

Depots  are  expected  to  give  customers  individual  attention  1 

OTHER 


Circle  the  item  that  you  feel  best  describes  your  expectations 
Aircraft  should  be  returned  from  MCAPP  in  less  tlian; 

120  days  100  days  80  days  50  days 

Aircraft  MCAPP  cost  should  be  less  than; 

$700K  $500K  $300K  $100K 


2  3  4  5 
2  3  4  5 

2  3  4  5 

2  3  4  5 

2  3  4  5 

2  3  4  5 

2  3  4  5 

2  3  4  5 

2  3  4  5 

2  3  4  5 

2  3  4  5 

2  3  4  5 


In  the  space  provided  below,  please  make  any  additional  comments  pertaining  to  your 
expectations  of  depot  services. 


PART  TWO:  PERFORMANCE 

This  portion  of  the  survey  deals  with  your  feelings  of  NADEP  North  Island’s  performance  as  an  offeror  of 
F/A-18  depot  level  maintenance.  Please  show  the  extent  to  which  you  believe  NADEP  NI’s  F/A-18 
maintenance  program  has  the  characteristics  described  in  each  statement.  Do  this  by  using  the  scale 
presented  below.  If  you  strongly  agree  that  NADEP  NI’s  F/A-18  maintenance  program  possess  that 
characteristic,  circle  the  number  five  on  the  same  line.  If  you  strongly  disagree  that  NADEP  NI’s  F/A-18 
maintenance  program  possess  that  characteristic,  circle  the  number  one  on  the  same  line.  If  your  feelings 
are  not  strong,  circle  the  number  between  one  and  five  that  best  reflects  your  feelings.  There  are  no  right 
or  wrong  answers  —  all  we  are  interested  in  is  a  number  that  best  shows  your  perception  about  NADEP 

NI’s  F/A-18  maintenance  program  performance. 

1=  Disagree  Strongly  2=Disagree  3=Undecided  4=Agree 

5=Agree  Strongly 

QUALITY 

Aircraft  delivered  to  squadrons  after  MCAPP  are  FMC 

1 

2 

3 

4  5 

Aircraft  delivered  from  depot  are  free  of  quality  defects 

1 

2 

3 

4  5 

Aircraft  delivered  from  MCAPP  have  a  “good”  paint  job 

1 

2 

3 

4  5 

Aircraft  logbooks  and  records  delivered  from  depot  are 
discrepancy  free 

1 

2 

3 

4  5 

Aircraft  delivered  from  depot  are  clean 

1 

2 

3 

4  5 

Aircraft  delivered  from  depot  are  FOD-ffee 

1 

2 

3 

4  5 

PACE  inspection  criteria  are  consistent 

1 

2 

3 

4  5 

All  modification  or  special  rework  requested  by  the  squadron  is 
accomplished  during  MCAPP 

1 

2 

3 

4  5 

TURNAROUND  TIME 

Depot  maintenance  does  not  impact  operational  requirements 

1 

2 

3 

4  5 

Aircraft  complete  depot  maintenance  on  schedule 

1 

2 

3 

4  5 

Customers  are  notified  immediately  of  changes  to  scheduled 
completion  dates 

1 

2 

3 

4  5 

COST 

My  activity  is  concerned  about  aircraft  depot  maintenance  costs 

1 

2 

3 

4  5 

Aircraft  MCAPP  costs  accurately  reflect  actual  depot  man-hours  expended 

1 

2 

3 

4  5 
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RELATIONSHIP 

My  depot  point-of-contact  is  easy  to  get  a  hold  of  1 

The  OPNAV,  NAVAIR  and  local  procedures  for  doing  business  1 

with  tire  depot  are  designed  for  easy  interaction  with  the  depot 

The  depot  responds  in  a  timely  manner  to  requests  for  services  1 

The  depot  responds  in  a  timely  manner  to  customer  complaints  1 

The  depot  informs  customers  of  the  specific  work  performed  on  1 

their  aircraft  as  part  of  MCAPP 

Aircraft  schedule  status  information  is  easy  to  obtain  1 

The  depot  visits  customers  often  enough  to  assess  their  needs  1 

The  depot  has  given  me  a  point  of  contact  to  facilitate  customer  1 

complaints/requests 


Depot  employees  deal  with  customers  in  a  courteous  and  helpful  manner  1 
The  depot  gives  customers  individual  attention  1 

OTHER 

Circle  the  item  that  you  feel  best  describes  NADEP  NI’s  performance: 

The  overall  quality  of  NADEP  NI’s  F/A-18  maintenance  program  is: 

Very  Poor  Poor  Fair  Good 

My  feelings  towards  NADEP  NTs  services  can  be  best  described  as: 

Very  Unsatisfied  Unsatisfied  Neutral  Satisfied 


2  3  4  5 
2  3  4  5 

2  3  4  5 
2  3  4  5 
2  3  4  5 

2  3  4  5 
2  3  4  5 
2  3  4  5 

2  3  4  5 
2  3  4  5 


Excellent 


Very  Satisfied 


In  the  space  provided  below,  please  make  any  additional  comments  pertaining  to  your 
feelings  of  NADEP  NI’s  performance  of  depot  services. 


PART  THREE:  IMPORTANCE 


This  portion  of  the  surv^  deals  with  your  assessment  about  the  importance  of  different  service 
characteristics  of  depot  maintenance.  A  five  means  you  consider  the  characteristic  very  important  for 
depots  performing  F/A-18  aircraft  maintenance  services,  a  one  means  it  is  very  unimportant.  You  should 
circle  the  number  between  one  and  five  that  best  reflects  your  assessment  of  the  importance  of  that 
characteristic.  There  are  no  right  or  wrong  answers  -  all  we  are  interested  in  is  a  number  that  best  shows 
how  important  each  characteristic  is  to  you  in  your  use  of  F/A-18  aircraft  depot  maintenance  services. 


1=  Very  Unimportant  2=Unimportant  3=Undecided  4-Important  5-Very  Important 


QUALITY 

Aircraft  being  returned  to  squadron  FMC  after  MCAPP 

Aircraft  being  delivered  from  depot  being  free  of  quality  defects 

Aircraft  being  delivered  from  MCAPP  with  “good”  paint  job 

Aircraft  logbooks  and  records  being  delivered  from  depot 
discrepancy  free 

Aircraft  being  delivered  from  depot  clean 

Aircraft  being  delivered  from  depot  FOD-free 

PACE  inspection  criteria  being  consistent 

Modification  or  special  rework  items  requested  by  the  squadron 
being  accomplished  during  MCAPP 

TURNAROUND  TIME 

Depot  maintenance  not  impacting  operational  requirements 

Aircraft  depot  maintenance  being  completed  on  schedule 

Customers  being  notified  immediately  of  changes  to  scheduled 
completion  dates 


1  2  3  4  5 
1  2  3  4  5 
1  2  3  4  5 
1  2  3  4  5 

1  2  3  4  5 
1  2  3  4  5 
1  2  3  4  5 
1  2  3  4  5 


1  2  3  4  5 
1  2  3  4  5 
1  2  3  4  5 


COST 

My  activity  being  concerned  about  aircraft  depot  maintenance  costs 
Aircraft  MCAPP  cost  being  related  to  actual  depot  man-hours  expended 
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RELATIONSHIP 

Depot  point-of-contact  being  easy  to  get  a  hold  of 

The  OPNAV,  NAVAIR  and  local  procedures  for  doing  business 
with  tlie  depot  being  designed  for  easy  interaction  with  the  depot 

The  depot's  response  time  to  requests  for  services 

The  depot's  response  time  to  customer  complaints 

The  depot  informing  customers  of  the  specific  work  being  performed 
on  their  aircraft  as  part  of  MCAPP 

Aircraft  schedule  status  information  being  easy  to  obtain 

The  depot  visiting  customers  periodically  to  assess  their  needs 

The  depot  providing  a  point  of  contact  to  facilitate  customer 
complaints/requests 

Depot  employees  dealing  with  customers  in  a  courteous  and 
helpful  manner 

Depots  giving  customers  individual  attention 

OTHER 

Please  rank  in  order  of  importance  (1  being  most  important,  etc)  the  following  considerations  as  they 
relate  to  depot  maintenance  of  F/A-18  aircraft; 

_  COST 

_  QUALITY 

_  TURNAROUND  TIME 

_  OTHER  (please  specify)  _ _ _ _ 

My  need  to  understand  the  depot’s  capabilities  and  constraints  is; 

Very  Unimportant  Unimportant  Undecided  Important  Very  Important 

In  the  space  provided  below,  please  make  any  additional  comments  about  what  you  feel  is 
important  about  depot  maintenance  of  naval  aircraft. 


1  2  3  4  5 

1  2  3  4  5 

1  2  3  4  5 

1  2  3  4  5 

1  2  3  4  5 

1  2  3  4  5 

1  2  3  4  5 

1  2  3  4  5 

1  2  3  4  5 

1  2  3  4  5 
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V.  CUSTOMER  PRETEST  QUESTIONNAIRE  RESULTS 


A.  INTRODUCTION 

In  this  chapter,  we  analyze  the  results  from  the  NADEP  NI  customer  pretest 
questionnaire.  This  analysis  shows  the  areas  on  which  NADEP  NI  must  focus  to  improve 
customer  satisfaction.  Analysis  was  conducted  by  customer  type  (squadron,  vvdng, 
TYCOM,  and  systems  command),  by  geographic  location  (east  vs.  west  coast)  and  by 
service  (Navy  vs.  Marine).  The  analysis  identifies  the  gaps  between  expectations  and 
performance  perceptions  of  the  different  NADEP  NI  F/A-18  customers.  It  then  identifies 
the  importance  that  customers  place  on  each  attribute.  WhUe  this  analysis  is  tailored  for 
NADEP  NI,  the  systematic  approach  can  be  used  by  all  service  depots  and  other 
government  agencies  as  well. 

B.  QUESTIONNAIRE  KEY 

To  assist  in  presenting  our  data,  we  developed  a  simple  set  of  alpha-numeric 
designators  to  identify  the  different  questions  on  the  pretest  questionnaire.  The 
designators  consisted  of  two  letters  and  a  number.  The  first  letter  designates  if  it  is  a 
question  from  the  expectations,  performance  or  importance  sections  of  the  questionnaire, 
using  the  letters  E,  P  or  I,  respectively.  The  next  letter  identifies  the  question  as  a  quality, 
turnaround,  cost  or  relations  attribute,  using  the  letters  Q,  T,  C  or  R,  respectively.  Finally 
the  number  identifies  the  order  it  appears  within  each  attribute.  For  example  the  alpha¬ 
numeric  designator  EQ4  identifies  the  fourth  question  under  the  attribute  quality  in  the 
expectations  section  of  the  pretest  questionnaire.  Figure  5.1  is  a  key  that  shows  the 
question  number  and  attribute  of  the  pretest  questionnaire  in  generic  form.  Placing  an  E,  P 
or  I  in  front  of  the  designator  would  give  the  section  of  the  questionnaire  being  analyzed. 
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PRETEST  QUESTIONNAIRE  DESIGNATOR  KEY 

Q1  Aircraft  delivered  to  squadrons  after  MCAPP  should  be  FMC 

Q2  Aircraft  delivered  from  depot  should  be  free  of  quality  defects 

Q3  Aircraft  delivered  from  MCAPP  should  have  a  “good”  paint  job 

Q4  Aircraft  logbooks  and  records  delivered  from  depot  should  be  discrepancy^  free 

Q5  Aircraft  delivered  from  depot  should  be  clean 

Q6  Aircraft  delivered  from  depot  should  be  FOD-free 

Q7  PACE  inspection  criteria  should  be  consistent 

Q8  All  modification  or  special  rework  items  requested  by  the  squadron  should  be  accomplished 
during  MCAPP 

T1  Depot  maintenance  should  not  impact  operational  requirements 
T2  Aircraft  should  complete  depot  maintenance  on  schedule 

T3  Customers  should  be  notified  immediately  of  changes  to  scheduled  completion  dates 
Cl  My  activity  should  be  concerned  about  aircraft  depot  maintenance  costs 
C2  Aircraft  MCAPP  cost  should  be  related  to  actual  depot  man-hours  expended 
R1  My  depot  point-of-contact  should  be  easy  to  get  a  hold  of 

R2  The  OPNAV,  NAVAIR  and  local  procedures  for  doing  business  with  the  depot  should  be 
designed  for  easy  interaction  with  the  depot 

R3  The  depot  should  respond  in  a  timely  maimer  to  requests  for  services 

R4  The  depot  should  respond  in  a  timely  manner  to  customer  complaints 

R5  The  depot  should  inform  customers  about  the  specific  work  performed  on  their  aircraft  as  part  of 

MCAPP 

R6  Aircraft  schedule  status  information  should  be  easy  to  obtain 

R7  The  depot  should  visit  customers  periodically  to  assess  their  needs 

R8  The  depot  should  provide  a  point  of  contact  to  facilitate  customer  complaints/requests 

R9  Depot  employees  should  deal  witli  customers  in  a  courteous  and  helpful  manner 

RIO  Depots  are  expected  to  give  customers  individual  attention _ 

Figure  5.1 
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c. 


KEY  TERMS  AND  ACRONYMS 


The  following  is  a  list  of  key  terms  and  acronyms  and  their  meanings  as  they 


pertain  to  this  analysis: 

Mean 

The  arithmetic  average  of  the  data  values.  (Greyer  &  Miller  1995) 

Median 

The  middle  value  for  a  data  set  ordered  in  magnitude.  (Greyer  & 
Miller  1995) 

Mode 

Most  frequently  used  response  in  data  set.  (Greyer  &  Miller  1995) 

Standard  Deviation 

A  measure  of  variability  obtained  as  the  square  root  of  the  number 
obtained  from  the  sum  of  squared  deviations  of  the  observations 
from  their  mean  divided  by  n-1.  (Greyer  &  Miller  1995) 

Variance 

The  square  of  the  standard  deviation.  (Greyer  &  Miller  1995) 

Expectations 

The  act  of  anticipating  an  occurrence  with  reason  or  justification. 
(Random  House  1992) 

Importance 

The  quality  of  having  much  significance  or  consequence. 

(Random  House  1992) 

Performance 

The  act  of  executing  an  act  in  the  established  manner.  (Random 
House  1992) 

Perceptions 

The  act  of  apprehending  by  means  of  the  senses  or  of  the  mind. 
(Random  House  1992) 

P-E  Gap 

The  difference  between  the  value  assigned  for  expectations  and  the 
value  assigned  for  performance  perceptions  of  a  customer  or  group 
of  customers.  (Parasuraman,  Zeithamal  and  Berry  1985) 

D.  VALUE  ASSIGNMENT 

Values  assigned  to  each  question  were  designated  by  compiling  all  responses  from 
customers  in  a  group  and  calculating  the  median,  mode,  mean,  variance  and  standard 
deviation  of  the  responses.  If  the  median  and  mode  were  the  same  value,  this  number  was 
used.  If  they  were  different,  whichever  value  was  closer  to  the  mean  was  used.  The 
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variance  and  standard  deviation  are  given  to  show  the  degree  of  diversity  of  responses 
from  the  customers.  Although  we  sent  questionnaires  to  100  percent  of  all  NADEP 
customers  who  had  interacted  vrith  NADEP  NT’s  F/A-18  maintenance  program  in  the  last 
18  months,  only  50  pretest  questionnaires  were  distributed.  The  statistical  measures  used 
in  this  analysis  assist  in  formulating  a  customer  satisfaction  measurement  system  for 
NADEP  NI  to  measure  customer  satisfaction.  Therefore,  due  to  the  small  sample  size,  we 
are  not  claiming  these  measures  to  be  of  statistical  significance. 


E.  CUSTOMER  TYPE  ANALYSIS 

The  first  analysis  was  conducted  by  customer  type.  This  analysis  shows  the  P-E 
gap  and  importance  factors  of  each  customer  type  and  compares  it  with  other  customer 
types.  We  began  by  analyzing  squadron  responses. 


1.  Squadron  Analysis 

Of  pretest  questionnaires  that  were  sent  to  squadrons,  a  total  of  65%  percent  of 
squadrons  responded.  The  following  chart  and  graph  show  the  expectation  responses 
received  from  Squadrons: 


SQUADRON  EXPECTATIONS 
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SQUADRON  EXPECTATIONS 


QUESTIONS 


The  results  of  the  “other”  section  of  the  expectations  part  of  the  questionnaire 
were  as  follows: 


Aircraft  should  be  returned from  MCAPP  in  less  than: 

#  of  days  %  of  Squadrons  with  response 


120  days  36% 

100  days  45% 

80  days  9% 

50  days  9% 


Aircraft  MCAPP  cost  should  be  less  than: 

SfOOO)  %  of  Squadrons  with  response 

$700K  18% 

$500K  36% 

$300K  36% 

$100K  10% 

It  should  be  noted  that  many  of  the  Squadrons’  written  comments  stated  that  they 

had  little  understanding  of  the  cost  issues  associated  with  MCAPP. 
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The  following  chart  and  graph  show  the  performance  perception  responses 


received  from  Squadrons: 


SQUADRON  PERFORMANCE  PERCEPTIONS 
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The  results  of  the  “other”  section  of  the  performance  perception  part  of  the  questionnaire 


were  as  follows: 


The  overall  quality  ofNADEP  NTs  F/A-18  maintenance  program  is: 


Response 


%  of  Squadrons  with  response 


Very  Poor  0% 

Poor  0% 

Fair  12% 

Good  62)% 

Excellent  25% 
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My  feelings  towards  NADEP  NI’s  services  can  be  best  described  as: 


ResDonse 

%  of  Sauadrons  with  resoonse 

Very  Unsatisfied 

0% 

Unsatisfied 

12% 

Neutral 

12% 

Satisfied 

76% 

Very  Satisfied 

0% 

The  following  chart  and  graph  show  the  importance  factor  responses  received 

fi'om  Squadrons: 

SQUADRON  IMPORTANCE  FACTORS 
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The  results  of  the  “other”  section  of  the  importance  factors  part  of  the 


questionnaire  were  as  follows: 

Please  rank  in  order  of  importance  the  following  considerations  as  they  relate  to  depot 
maintenance  ofF/A-18  aircraft:  (Cost,  Quality,  Turnaround  Time) 

91%  of  Squadrons  responding  ranked  quality  first,  turnaround  time  second  and  cost  third. 
9%  of  Squadrons  responding  ranked  quality  first,  cost  second,  and  turnaround  time  third. 
My  need  to  understand  the  depot ’s  capabilities  and  constraints  is: 

Response  %  of  Squadrons  with  response 


Very  Unimportant 
Unimportant 
Undecided 
Important 
Very  Important 


9% 

0% 

0% 

64% 

27% 


The  following  chart  and  graph  represent  the  results  of  conducting  a  P-E 
gaps/importance  factor  analysis  from  squadron  responses: 


SQUADRON  P-E  GAPS/EMPORTANCE  ANALYSIS 
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SQUADRON  P-E  GAP/IMPORTANCE  ANALYSIS 


QUESTIONS 


IMPORTANCE 


♦1 >2 a3x4x5 


To  best  improve  customer  service,  we  felt  that  any  response  with  a  P-E  gap  of 
more  than  1  and  an  importance  factor  of  4  or  5  required  the  most  immediate  attention. 
P-E  gaps  of  1  or  less  generally  signify  satisfaction  with  the  attribute,  with  some  possibility 
for  improvement.  The  results  of  this  analysis  show  that  in  order  to  improve  customer 
satisfaction  among  squadrons,  NADEP  NI  should  focus  its  efforts  on  the  consistency  of 
PACE  inspections  and  notifying  customers  about  any  schedule  changes.  Other  areas  of 
improvement  should  include  the  depot’s  response  time  to  customer  complaints  and 
informing  customers  of  the  specific  work  being  performed  on  their  aircraft  as  part  of 
MCAPP.  It  should  be  noted,  however,  that  squadrons  felt  that  NADEP  NI  does  an 
excellent  job  of  providing  a  point  of  contact  to  facilitate  customer  complaints/requests 
(zero  P-E  gap). 

Overall,  the  squadrons  seem  to  be  satisfied  with  the  services  they  have  been 
receiving  from  NADEP  NI.  They  seem  content  with  the  current  turnaround  time  of  108 
days,  as  long  as  they  are  notified  immediately  of  changes,  so  they  can  adjust  their 
operational  schedules.  One  attribute  that  had  a  P-E  gap  of  two  and  an  importance  factor 
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of  four,  was  that  all  modifications  or  special  rework  items  requested  by  the  squadron 
should  be  accomplished  during  MCAPP.  However,  many  comments  from  the  squadrons 
stated  that  they  did  not  understand  what  capabilities  or  constraints  affect  the  depot’s 
ability  to  complete  such  a  request.  This  fact,  along  with  the  large  percent  of  squadrons 
who  reported  that  it  was  important  to  understand  NADEP  NI’s  capabilities  and 
constraints,  points  to  the  need  to  educate  customers  about  the  depot’s  capabilities  and 
constraints,  if  customers  are  to  have  realistic  service  expectations. 

2.  Wing  Analysis 

Of  the  pretest  questionnaires  that  were  sent  to  Wing  staffs,  a  total  of  50  percent 
responded.  We  sent  multiple  pretest  questionnaires  to  each  of  four  Wing  staffs  (two  Navy, 
two  Marine  Corps).  We  received  at  least  one  response  from  each  of  the  four  Wings.  The 
following  chart  and  graph  show  the  expectation  responses  received  from  Wings: 


WING  EXPECTATIONS 


QUESTION 

^g 

m 

gg 

E3 

SI 

IS 

SISl 

gg 

ui 

ER10 

MEDIAN 

H 

B 

B 

B 

B 

B 

B 

B 

B 

B 

B 

B 

B 

B 

B 

B 

B 

B 

B 

B 

B 

B 

H 

MODE 

H 

B 

B 

B 

B 

B 

B 

B 

B 

B 

B 

B 

B 

B 

B 

B 

B 

B 

B 

B 

B 

B 

Bi 

MEAN 

la 

B 

B 

B 

m 

B 

B 

B 

B 

B 

B 

im 

B 

B 

B 

B 

B 

B 

B 

B 

B 

B 

VARIANCE 

H 

B 

B 

B 

m 

B 

B 

B 

^gjj 

B 

B 

gg 

B 

gg 

B 

B 

^g 

B 

iimigiiii 

■ 

B 

B 

B 

B 

B 

m 

E9 

1^2 

1 

■ 

i^i 

ggj 

gg 

B 

gg 

B 

B 

B 

BSi 

84 


5-1 

*9  4 


(0 

in 

CO 

z 

o 

Q. 

CO 

UJ 


WING  EXPECTATIONS 


— 1 

i 

i  EC 

l-EI 

1 

d 

n 

n 

n 

m 

ns 

i 

► 

L  CD 

n 

7 

n 

- E 

a 

i 

Wo 

^  EC 

1 

ER10 

ER10 


#  MEDIAN 
■  MODE 
A  MEAN 


0  1  2  3  4  5  6  7  8  9  10  11  12  13  14  15  16  17  18  19  20  21  22  23 

QUESTIONS 


The  results  of  the  “other”  section  of  the  expectations  part  of  the  questionnaire 
were  as  follows: 

Aircraft  should  be  returned from  MCAPP  in  less  than: 


#  of  days  %  of  Wings  with  response 


120  days 

50% 

100  days 

50% 

80  days 

0% 

50  days 

0% 

Aircraft  MCAPP  cost  should  be  less  than: 

SfOOOl 

%  of  Wings  with  response 

$700K 

0% 

$500K 

50% 

$300K 

50% 

$100K 

0% 

Some  of  the  written  in  comments  by  the  Wings  stated  that  they  felt  that  cost  should  be  tied 
to  the  work  done  and  depot  overhead  cost  should  be  funded  separately. 
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The  following  chart  and  graph  show  the  performance  perception  responses 
received  from  Wings: 

WING  PERFORMANCE  PERCEPTIONS 


The  results  of  the  “other”  section  of  the  performance  perception  part  of  the 
questionnaire  were  as  follows: 

The  overall  quality  of  NADEP  M’s  F/A-18  maintenance  program  is: 

ResDonse  %  of  Wines  with  response 


Very  Poor  0% 

Poor  0% 

Fair  0% 

Good  100% 

Excellent  0% 
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My  feelings  towards  NADEP  NI’s  services  can  be  best  described  as: 


Response 


%  of  Wings  with  response 


Very  Unsatisfied  0% 

Unsatisfied  0% 

Neutral  25% 

Satisfied  75% 

Very  Satisfied  0% 


The  following  chart  and  graph  show  the  importance  factor  responses  received 
from  Wings: 


WING  IMPORTANCE  FACTORS 
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The  results  of  the  “other”  section  of  the  importance  factors  part  of  the 


questionnaire  were  as  follows: 

Please  rank  in  order  of  importance  the  following  considerations  as  they  relate  to  depot 

maintenance  ofF/A-18  aircraft:  (Cost,  Quality,  Turnaround  Time) 

75%  of  Wings  responding  ranked  quality  first,  turnaround  time  second  and  cost  third. 

25%  of  Wings  responding  ranked  quality  first,  cost  second,  and  turnaround  time  third. 

My  need  to  understand  the  depot ’s  capabilities  and  constraints  is: 

Response  %  of  Wm2s  with  response 

Very  Unimportant  0% 

Unimportant  0% 

Undecided  0% 

Important  25% 

Very  Important  75% 

The  following  chart  and  graph  represent  the  results  of  conducting  a  P-E 

gaps/importance  factor  analysis  from  Wing  responses; 

WING  P-E  GAPS/IMPORTANCE  ANALYSIS 
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WING  P-E  GAP/I WIPORTANCE  ANALYSIS 
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QUESTIONS 


IMPORTANCE  ♦1b2a3x4  5k5 


The  results  of  this  analysis  show  that  in  order  to  improve  customer  satisfaction 
among  Wings,  as  with  Squadrons,  NADEP  NI  should  focus  its  efforts  on  the  consistency 
of  PACE  inspections.  Wings  also  were  concerned  about  all  of  the  turnaround  time 
attributes.  They  felt  there  was  room  for  improvement  in  not  impacting  on  operational 
requirements  and  completeing  aircraft  on  schedule,  and  to  a  lesser  degree  being  notified  of 
schedule  changes.  Other  areas  of  improvement  included  Foreign  object  damage  (FOD), 
fewer  quality  defects  and  a  better  paint  job.  As  noted  earlier,  several  respondents  felt  that 
cost  should  be  associated  with  actual  work  completed  on  the  aircraft. 

Overall,  the  wings  seem  to  be  satisfied  with  the  services  they  have  been  receiving 
from  NADEP  NI.  They  seem  fairly  content  with  the  current  turnaround  time  of  108  days. 
However,  half  of  the  respondents  felt  this  could  be  improved  All  wing  respondents  felt 
that  it  was  important  or  very  important  for  them  to  understand  NADEP  NT’s  capabilities 
and  constraints.  Once  again,  this  points  to  the  need  to  educate  customers  about  depot 
capabilities  and  constraints,  if  the  depot  wishes  its  customers  to  have  realistic  service 
expectations. 
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3.  TYCOM  Analysis 

Of  the  pretest  questionnaires  that  were  sent  to  TYCOMs,  a  total  of  66  percent 
responsed.  However,  as  with  the  wings,  we  received  at  least  two  responses  from  each 
TYCOM  who  was  sent  a  questionnaire.  The  following  chart  and  graph  show  the 
expectation  responses  received  from  TYCOMs: 

TYCOM  EXPECTATIONS 
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TYCOM  EXPECTATIONS 


QUESTIONS 


The  results  of  the  “other”  section  of  the  expectations  part  of  the  questionnaire 
were  as  follows: 

Aircraft  should  be  returned from  MCAPP  in  less  than: 

#  of  days  %  of  TYCOMs  with  response 

120  days  25% 

100  days  50% 

80  days  25% 

50  days  0% 
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Aircraft  MCAPP  cost  should  be  less  than: 


SfOOO)  %  of  Wings  with  response 


$700K 

0% 

$500K 

75% 

$300K 

25% 

$100K 

0% 

The  following  chart  and  graph  show  the  performance  perception  responses 


received  from  TYCOMs; 

TYCOM  PERFORMANCE  PERCEPTIONS 
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QUESTIONS 


The  results  of  the  “other”  section  of  the  performance  perception  part  of  the 
questionnaire  were  as  follows: 
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The  overall  quality  of  NADEP  NTs  F/A-18  maintenance  program  is: 


Response 


%  of  TYCOMs  with  response 


Very  Poor  0% 

Poor  0% 

Fair  50% 

Good  0% 

Excellent  50% 


My  feelings  towards  NADEP  NI 's  services  can  he  best  described  as: 


Response 


Very  Unsatisfied 
Unsatisfied 
Neutral 
Satisfied 
Very  Satisfied 


0% 

0% 

50% 

0% 

50% 


The  following  chart  and  graph  show  the  importance  factor  responses  received 


from  TYCOMs: 


TYCOM  IMPORTANCE  FACTORS 


lagaaBSBaBBaBBMBBBBBBM 


VARIANCE  1  3.3  0.3 


The  results  of  the  “other”  section  of  the  importance  factors  part  of  the 
questionnaire  were  as  follows; 

Please  rank  in  order  of  importance  the  following  considerations  as  they  relate  to  depot 

maintenance  of  F/A- 18  aircraft:  (Cost,  Quality,  Turnaround  Time) 

75%  of  TYCOMs  responding  ranked  quality  first,  turnaround  time  second  and  cost  third. 

25%  of  TYCOMs  responding  ranked  quality  first,  cost  second,  and  turnaround  time  third. 

My  need  to  understand  the  depot ’s  capabilities  and  constraints  is: 

Response  %  of  TYCOMs  with  response 

Very  Unimportant  0% 

Unimportant  0% 

Undecided  0% 

Important  25% 

Very  Important  75% 
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The  following  chart  and  graph  represent  the  results  of  conducting  a  P-E 


gaps/importance  factor  analysis  from  TYCOM  responses: 

TYCOM  P-E  GAPS/IMPORTANCE  ANALYSIS 
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The  results  of  this  analysis  show  that  to  improve  customer  satisfaction  among 
TYCOMs,  as  with  Wings  and  Squadrons,  NADEP  NI  should  focus  its  efforts  on  the 
consistency  of  PACE  inspections.  TYCOMs  are  also  concerned  about  all  the  attributes  of 
turnaround  time.  They  especially  felt  there  was  room  for  improvement  in  completing 
aircraft  on  schedule.  Other  areas  of  improvement  include  the  aircraft  having  fewer  quality 
defects  and  delivering  discrepancy  free  aircraft  logbooks  and  records. 

There  was  a  definite  split  among  the  TYCOMs  about  whether  they  were  satisfied 
with  the  services  they  have  received  from  NADEP  NI.  Fifty  percent  seemed  to  be 
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extremely  satisfied,  while  the  other  fifty  percent  seemed  to  be  neutral  on  the  subject.  Of 
all  customer  types,  the  TYCOM’ s  seemed  the  least  content  with  the  current  turnaround 
time  of  108  days,  with  75%  of  all  respondents  stating  this  standard  could  be  improved. 

All  respondents  felt  that  it  was  important  or  very  important  for  them  to  understand  the 
capabilities  and  constraints  of  NADEP  NI.  This  again  points  to  the  need  to  educate 
customers  about  the  depot’s  capabilities  and  constraints,  if  the  depot  wishes  its  customers 
to  have  realistic  service  expectations. 

4.  NAVAER  Analysis 

Pretest  questionnaire  were  sent  to  offices  within  NAVAIR  Code  6.0,  Deputy 
Director  for  Operations,  and  NAVAIR  3 . 1 . 1 . 1 C,  the  F/A- 1 8  Assistant  Program  Manager, 
Logistics  (F/A- 18  APML).  Of  the  pretest  questionnaires  sent  to  NAVAIR,  a  total  of  50 
percent  responded.  The  following  chart  and  graph  show  the  expectation  responses 
received  from  NAVAIR; 
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The  results  of  the  “other”  section  of  the  expectations  part  of  the  questionnaire 
were  as  follows: 

Aircraft  should  be  returned from  MCAPP  in  less  than:  100  DAYS 
Aircraft  MCAPP  cost  should  be  less  than:  $500K 

The  following  chart  and  graph  show  the  performance  perception  responses 
received  from  NAVAIR: 


NAVAIR  PERFORMANCE  PERCEPTIONS 


The  results  of  the  “other”  section  of  the  performance  perception  part  of  the 
questionnaire  were  as  follows: 

The  overall  quality  of  NADEP  NI  ’s  F/A-18  maintenance  program  is:  EXCELLENT 
My  feelings  towards  NADEP  NI’s  services  can  be  best  described  as:  VERY  SATISFIED 


The  following  chart  and  graph  show  the  importance  factor  responses  received 


fromNAVAIR; 


NAVAIR  IMPORTANCE  FACTORS 


QUESTION 

[Q| 

|g| 

E3 

13 

13 

^1 

m 

m 

m 

13 

13 

13 

01 

02 

m 

m 

IR10 

RESPONSE 

B 

B 

B 

B 

B 

B 

B 

B 

B 

B 

B 

B 

B 

B 

B 

B 

B 

B 

B 

B 

B 

B 

4 

NAVAIR  IMPORTANCE  FACTORS 


QUESTIONS 


The  results  of  the  “other”  section  of  the  importance  factors  part  of  the 
questionnaire  were  as  follows: 

Please  rank  in  order  of  importance  the  following  considerations  as  they  relate  to  depot 
maintenance  of  F/A- 18  aircraft:  (Cost,  Quality,  Turnaround  Time) 

Cost  first,  Turnaround  Time  second,  Quality  third 

My  need  to  understand  the  depot ’s  capabilities  and  constraints  is:  IMPORTANT 
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The  following  chart  and  graph  represent  the  results  of  conducting  a  P-E 


gaps/importance  factor  analysis  from  NAVAIR: 

NA  VATR  P-E  GAPS/IMPORTANCE  ANALYSIS 
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♦  1  *2  a3  x4  x5 


The  results  of  this  analysis  show  that  NAVAIR  had  no  P-E  gaps  greater  than  one. 
There  were  only  three  attributes  with  a  P-E  gap  of  1  and  an  importance  factor  of  five. 
They  are  consistent  PACE  inspection  criteria,  depot  maintenance  not  impacting 
operational  requirements,  and  aircraft  completing  depot  maintenance  on  schedule.  These 
would  be  the  areas  that  NADEP  NI  should  focus  on  to  improve  NAVAIR’ s  customer 
satisfaction.  This  analysis  shows  that  NAVAIR  is  very  satisfied  with  NADEP  NI’s 
performance.  There  were  three  attributes  that  actually  had  a  negative  P-E  gap.  The  other 
interesting  fact  is  that  NAVAIR  ranked  cost  first  in  importance  and  quality  third.  They 
were  the  only  customer  not  to  rank  quality  first. 
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F.  GEOGRAPHIC  ANALYSIS 


The  second  analysis  was  conducted  by  location  of  customer.  Customers  were 
grouped  into  east  or  west  coast  according  to  their  location.  For  the  purpose  of  this 
analysis,  NAVAIR’s  response  was  omitted  as  was  the  response  from  the  Naval  Flight 
Demonstration  Squadron.  This  analysis  shows  the  P-E  gap  and  importance  factors  of 
customers  located  on  each  coast  and  compares  them  with  one  another.  We  began  by 
analyzing  east  coast  responses. 


1.  East  Coast  Analysis 

The  following  chart  and  graph  show  the  expectation  responses  received  from 
customers  on  the  east  coast: 
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The  results  of  the  “other”  section  of  the  expectations  part  of  the  questionnaire 


were  as  follows: 


Aircraft  should  be  returned from  MCAPP  in  less  than: 


#  of  days  %  of  East  Coast  with  response 


120  days  50% 

100  days  38% 

80  days  12% 

50  days  0% 


Aircraft  MCAPP  cost  should  be  less  than: 


$(0001  %  of  East  Coast  with  response 


$700K 

SSOOK 

$300K 

SIOOK 


12% 

50% 

38% 

0% 


The  following  chart  and  graph  show  the  performance  perception  responses 
received  from  east  coast  customers: 

EAST  COAST  PERFORMANCE  PERCEPTIONS 
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The  results  of  the  “other”  section  of  the  performance  perception  part  of  the 
questionnaire  were  as  follows: 

The  overall  quality  of  NADEP  NI’s  F/A-18  maintenance  program  is: 


Response 


%  of  East  Coast  with  response 


Very  Poor 

0% 

Poor 

0% 

Fair 

33% 

Good 

50% 

Excellent 

17% 

My  feelings  towards  NADEP  NI  ’s  senices  can  be  best  described  as: 


Response 


%  of  East  Coast  with  response 


Very  Unsatisfied  0% 

Unsatisfied  0% 

Neutral  50% 

Satisfied  50% 

Very  Satisfied  0% 
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The  following  chart  and  graph  show  the  importance  factor  responses  received 


from  east  coast  customers: 

EAST  COAST  IMPORTANCE  FACTORS 
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The  results  of  the  “other”  section  of  the  importance  factors  part  of  the 
questionnaire  were  as  follows: 

Please  rank  in  order  of  importance  the  following  considerations  as  they  relate  to  depot 
maintenance  of  F/ A- 18  aircraft:  (Cost,  Quality,  Turnaround  Time) 

87%  of  east  coast  responses  ranked  quality  first,  turnaround  time  second  and  cost  third. 
13%  of  east  coast  responses  ranked  quality  first,  cost  second,  and  turnaround  time  third. 


My  need  to  understand  the  depot ’s  capabilities  and  constraints  is: 


ResDonse 

%  of  East  Coast  w; 

Very  Unimportant 

0% 

Unimportant 

0% 

Undecided 

0% 

Important 

63% 

Very  Important 

37% 

The  following  chart  and  graph  represent  the  results  of  conducting  a  P-E 
gaps/Importance  factor  analysis  from  east  coast  responses: 
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The  results  of  this  analysis  show  that  in  order  to  improve  customer  satisfaction 
among  east  coast  customers,  NADEP  NI  should  focus  its  efforts  on  ensuring  aircraft 
complete  MCAPP  on  schedule,  improving  the  consistency  of  PACE  inspections,  ensuring 
aircraft  are  free  of  quality  defects  and  are  FOD-free.  East  coast  customers  had  a  total  of 
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ten  attributes  with  a  P-E  gap  greater  than  one,  with  five  of  them  having  an  importance 
factor  of  five. 

Overall,  east  coast  customers  seem  to  be  neutral  to  satisfied  with  NADEP  NT’s 
services.  They  seem  content  with  the  current  turnaround  time  of  108  days,  as  long  as 
aircraft  complete  MCAPP  as  scheduled.  Other  attributes  that  need  improvment  are 
accomplishing  the  modifications  or  special  rework  requested  during  MCAPP,  improving 
procedures  for  doing  business  with  the  depot,  informing  the  customers  of  the  specific 
work  performed  during  MCAPP  and  the  depot  visiting  customers  often  enough  to  assess 
their  needs. 


2.  West  Coast  Analysis 

The  following  chart  and  graph  show  the  expectation  responses  received  fi'om 
customers  on  the  west  coast: 

WEST  COAST  EXPECTATIONS 
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The  results  of  the  “other”  section  of  the  expectations  part  of  the  questionnaire 
were  as  follows: 


$700K 

$500K 

$300K 

$100K 


10% 

50% 

30% 

10% 
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The  following  chart  and  graph  show  the  performance  perception  responses 
received  from  west  coast  customers; 

WEST  COAST  PERFORMANCE  PERCEPTIONS 
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The  results  of  the  “other”  section  of  the  performance  perception  part  of  the 
questionnaire  were  as  follows: 

The  overall  quality  of  NADEP  NI’s  F/A-18  maintenance  program  is: 


Response 


%  of  West  Coast  with  response 


Very  Poor  0% 

Poor  0% 

Fair  11% 

Good  56% 

Excellent  33% 
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My  feelings  towards  NADEP  NI’s  services  can  he  best  described  as: 


Response 


%  of  West  Coast  with  response 


Very  Unsatisfied  0% 

Unsatisfied  11% 

Neutral  11% 

Satisfied  56% 

Very  Satisfied  22% 


The  following  chart  and  graph  show  the  importance  factor  responses  received 
from  west  coast  customers: 

WEST  COAST  IMPORTANCE  FACTORS 
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The  results  of  the  “other”  section  of  the  importance  factors  part  of  the 
questionnaire  were  as  follows: 

Please  rank  in  order  of  importance  the  following  considerations  as  they  relate  to  depot 
maintenance  of  F/A-18  aircraft:  (Cost,  Quality,  Turnaround  Time) 

80%  of  west  coast  responses  ranked  quality  first,  turnaround  time  second  and  cost  third. 
20%  of  west  coast  responses  ranked  quality  first,  cost  second,  and  turnaround  time  third. 


My  need  to  understand  the  depot’s  capabilities  and  constraints  is: 


Response 

%  of  West  Coast  M 

Very  Unimportant 

0% 

Unimportant 

0% 

Undecided 

0% 

Important 

40% 

Very  Important 

60% 

The  following  chart  and  graph  represent  the  results  of  conducting  a  P-E 
gaps/importance  factor  analysis  from  west  coast  responses; 
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WEST  COAST  P-E  GAP/IMPORTANCE  ANALYSIS 


QUESTIONS 


IMPORTANCE 


♦  1  *2  a3  x4  x5 


It  is  important  to  note  that  west  coast  customers  only  had  three  attributes  with  a  P- 
E  gap  greater  than  one,  with  none  of  them  having  an  importance  factor  of  five.  The  results 
of  this  analysis  show  that  to  improve  customer  satisfaction  among  west  coast  customers, 
NADEP  NI  should  focus  its  efforts  on  improving  the  attributes  of  completing  the 
modifications  or  special  rework  requested  during  MCAPP  and  ensuring  that  depot 
maintenance  does  not  impact  operational  requirements. 

West  coast  customers  seem  to  be  satisfied  to  very  satisfied  with  the  NADEP  NT’s 
services.  Overall,  the  west  coast  seemed  to  have  a  much  higher  level  of  customer 
satisfaction  than  the  east  coast.  This  could  be  due  to  the  time  difference.  The  east  coast 
had  a  half  a  point  larger  P-E  gap  in  both  the  response  time  for  services  and  complaints. 
NADEP  NI  is  on  the  west  coast,  with  basically  the  same  operating  hours  as  customers  on 
the  west  coast,  which  provides  more  time  for  communication  and  interaction. 

Another  significant  difference  was  the  P-E  gap  for  aircraft  completing  depot 
maintenance  on  schedule.  The  east  coast  had  a  P-E  gap  of  3,  while  the  west  coast  only 
had  a  P-E  gap  of  .5.  This  needs  to  be  investigated  to  see  if  this  is  truly  a  performance 
problem,  or  if  east  coast  customers  have  unrealistic  scheduling  expectations. 
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G.  SERVICE  ANALYSIS 


The  final  analysis  was  conducted  by  service  of  customer.  Wings  and  Squadrons 
were  grouped  according  to  their  service,  Navy  or  Marine.  For  the  purpose  of  this  analysis 
NAVAIR  and  TYCOM  responses  were  omitted  since  they  support  both  services.  This 
analysis  shows  the  P-E  gap  and  importance  factors  of  customers  in  the  Navy  and  Marines 
and  compares  each  of  them  with  one  another.  We  begin  by  analyzing  Navy  responses. 


1.  Navy  Analysis 

The  following  chart  and  graph  show  the  expectation  responses  received  from 
customers  in  the  Navy: 
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The  results  of  the  “other”  section  of  the  expectations  part  of  the  questionnaire 


were  as  follows: 

Aircraft  should  be  returned from  MCAPP  in  less  than: 


#  of  davs 

%  of  Naw  with  resoonse 

120  days 

40% 

100  days 

40% 

80  days 

10% 

50  days 

10% 

Aircraft  MCAPP  cost  should  be  less  than: 

StOOOl 

%  of  Naw  with  resoonse 

$700K 

10% 

$500K 

30% 

$300K 

50% 

$100K 

10% 

The  following  chart  and  graph  show  the  performance  perception  responses 
received  from  Navy  customers; 

NAVY  PERFORMANCE  PERCEPTIONS 
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NAVY  PERFORMANCE  PERCEPTIONS 
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QUESTIONS 


The  results  of  the  “other”  section  of  the  performance  perception  part  of  the 
questionnaire  were  as  follows: 

The  overall  quality  of  NADEP  NI’s  F/A-18  maintenance  program  is: 


ResDonse 

%  of  Naw  with  response 

Very  Poor 

0% 

Poor 

0% 

Fair 

12% 

Good 

75% 

Excellent 

13% 

My  feelings  towards  NADEP  NTs  services  can  be  best  described  as: 

Response  %  of  Naw  with  response 

Very  Unsatisfied 

0% 

Unsatisfied 

12% 

Neutral 

13% 

Satisfied 

75% 

Very  Satisfied 

0% 
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The  following  chart  and  graph  show  the  importance  factor  responses  received 
from  Navy  customers; 


NAVY  IMPORTANCE  FACTORS 
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The  results  of  the  “other”  section  of  the  importance  factors  part  of  the 
questionnaire  were  as  follows; 

Please  rank  in  order  of  importance  the  following  considerations  as  they  relate  to  depot 
maintenance  of  F/A-18  aircraft:  (Cost,  Quality,  Turnaround  Time) 

80%  of  Navy  responses  ranked  quality  first,  turnaround  time  second  and  cost  third. 
20%  of  Navy  responses  ranked  quality  first,  cost  second,  and  turnaround  time  third. 
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My  need  to  understand  the  depot 's  capabilities  and  constraints  is: 


Response 

%  of  Naw  with  res 

Very  Unimportant 

10% 

Unimportant 

0% 

Undecided 

0% 

Important 

50% 

Very  Important 

40% 

The  following  chart  and  graph  represent  the  results  of  conducting  a  P-E 
gaps/importance  factor  analysis  from  Navy  responses: 

NAVY  P-E  GAPS/IMPQRTANCE  ANALYSIS 
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IMPORTANCE 


♦  1  *2  a3x4)k5 


The  results  of  this  analysis  show  that  in  order  to  improve  customer  satisfaction 
among  Navy  customers,  NADEP  NI  should  ensure  aircraft  complete  MCAPP  on  schedule 
and  immediately  notify  customers  of  changes  to  scheduled  completion  dates.  Navy 
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customers  had  a  total  of  six  attributes  with  a  P-E  gap  greater  than  one,  two  of  them 
having  an  importance  factor  of  five. 

Overall,  Navy  customers  seem  to  be  satisfied  with  the  NADEP  NT’s  services. 

They  seem  content  with  the  current  turnaround  time  of  108  days,  as  long  as  aircraft 
complete  MCAPP  as  scheduled.  The  other  attributes  that  need  some  improvement  are  the 
depot  responding  in  a  timely  manner  to  requests  for  customer  complaints  and  informing 
customers  of  the  specific  work  performed  on  their  aircraft  as  part  of  MCAPP. 


2.  Marine  Analysis 

The  following  chart  and  graph  show  the  expectation  responses  received  from 
Marine  customers: 
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The  results  of  the  “other”  section  of  the  expectations  part  of  the  questionnaire 
were  as  follows: 

Aircraft  should  be  returned from  MCAPP  in  less  than: 

#  of  days  %  of  Marines  with  response 

120  days  40% 

100  days  60% 

80  days  0% 

50  days  0% 


Aircraft  MCAPP  cost  should  be  less  than: 

S(OOO)  %  of  Marines  with  response 


$700K 

20% 

$500K 

60% 

$300K 

20% 

$100K 

0% 

The  following  chart  and  graph  show  the  performance  perception  responses 


received  from  Marine  customers; 

MARINE  PERFORMANCE  PERCEPTIONS 
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The  results  of  the  “other”  section  of  the  performance  perception  part  of  the 
questionnaire  were  as  follows; 

The  overall  quality  of  NADEP  NI ’s  F/A-18  maintenance  program  is: 


Veiy  Poor 

Poor 

Fair 

Good 

Excellent 


0% 

0% 

0% 

75% 

25% 


My  feelings  towards  NADEP  NI’s  services  can  be  best  described  as: 


Response 


Very  Unsatisfied 
Unsatisfied 
Neutral 
Satisfied 
Very  Satisfied 


%  of  Marines  with  response 


0% 

0% 

25% 

75% 

0% 
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The  following  chart  and  graph  show  the  importance  factor  responses  received 


from  Marine  customers; 


MARINE  IMPORTANCE  FACTORS 
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The  results  of  the  “other”  section  of  the  importance  factors  part  of  the 


questionnaire  were  as  follows: 


Please  rank  in  order  of  importance  the  following  considerations  as  they  relate  to  depot 
maintenance  of  F/A- 18  aircraft:  (Cost,  Quality,  Turnaround  Time) 

100%  of  Marine  responses  ranked  quality  first,  turnaround  time  second  and  cost  third. 
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My  need  to  understand  the  depot 's  capabilities  and  constraints  is: 


ResDonse 

%  of  Marines  with 

Very  Unimportant 

0% 

Unimportant 

0% 

Undecided 

0% 

Important 

60% 

Very  Important 

40% 

The  following  chart  and  graph  represent  the  results  of  conducting  a  P-E 
gaps/importance  factor  analysis  from  Marine  responses: 


MARINE  P-E  GAPS/IMPORTANCE  ANALYSIS 
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The  results  of  this  analysis  show  that  in  order  to  improve  customer  satisfaction 
among  Marines,  NADEP  NI  should  immediately  focus  its  efforts  on  improving  the 
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consistency  of  PACE  inspections  and  informing  customers  about  specific  work  performed 
on  their  aircraft  as  part  of  MCAPP.  Other  attributes  of  concern  were  modifications 
requested  by  the  squadron  being  accomplished  during  MCAPP,  MCAPP  cost  accurately 
reflecting  actual  depot  man-hours  expended,  aircraft  being  delivered  FOD-free  and  aircraft 
completing  depot  maintenance  on  schedule.  Marine  customers  had  eleven  attributes  with  a 
P-E  gap  greater  than  one,  with  eight  of  them  having  an  importance  factor  of  five. 

Marine  customers  seem  to  be  satisfied  with  the  services  they  receive  from  NADEP 
NI.  Overall,  the  Navy  seemed  to  have  a  much  higher  level  of  customer  satisfaction  than 
the  Marines.  However,  upon  analyzing  the  data,  this  seems  to  be  a  function  of  Marines 
having  higher  expectations  rather  than  lower  performance  perceptions  of  NADEP  NI. 

H.  CONCLUSION 

The  following  table  shows  the  P-E  gaps/importance  analysis  of  all  customer 
groups.  The  P-E  gaps  that  are  underlined  have  an  importance  factor  of  five.  The  P-E 
gaps  that  are  italicized  have  an  importanct  factor  of  four.  P-E  gaps  in  regular  type  have 
and  importance  factor  of  three.  (There  were  no  importance  factors  below  three) 


CUSTOMER  GROUP  P-E  GAPS/IMPORTANCE  ANALYSIS  SUMMARIZATION 
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The  following  summarizes,  by  customer  groups,  the  most  important  attributes  on 
which  NADEP  NI  must  focus  to  improve  customer  satisfaction: 

Consistency  of  PACE  inspections 
Notifying  customers  about  any  schedule  changes 


Consistency  of  PACE  inspections 

Depot  maintenance  not  impacting  operational  requirements 

Completing  depot  maintenance  of  aircraft  on  schedule 


TYCOM: 

*  Consistency  of  PACE  inspections 

*  Completing  depot  maintenance  of  aircraft  on  schedule 


NAVAIR: 

*  Consistency  of  PACE  inspections 


East  Coast: 

*  Consistency  of  PACE  inspections 

*  Completing  depot  maintenance  of  aircraft  on  schedule 
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Ensuring  aircraft  are  free  of  quality  defects 
Ensuring  aircraft  are  FOD  free 


* 

* 


Depot  maintenance  not  impacting  operational  requirements 
Completing  all  requested  modification  or  special  rework  during  MCAPP 

Navy: 

Completing  depot  maintenance  of  aircraft  on  schedule 
Notifying  customers  about  any  schedule  changes 

Consistency  of  PACE  inspections 

Informing  customers  about  specific  work  performed  on  their  aircraft 

The  P-E  gaps/importance  analysis  was  an  excellent  analytical  tool  to  develop  a 
tailored  set  of  customer  satisfaction  metrics  for  NADEP  NI.  It  provided  the  means  to 
analyze  data  from  different  customer  points  of  view,  which  helped  focus  our  efforts.  This 
method  would  be  very  beneficial  for  other  service  depots  and  government  agencies  who 
wish  to  measure  customer  satisfaction.  The  key  element,  as  noted  previously,  is  the 
necessity  to  educate  the  customer  about  the  NADEP’ s  various  capabilities  and 
constraints  in  order  for  the  customer  to  develop  realistic  expectations. 


Mannes: 

* 


* 


West  Coast: 
* 


* 
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VI.  A  PROPOSED  CUSTOMER  SATISFACTION  MEASUREMENT  SYSTEM 

FOR  NADEP  NORTH  ISLAND 


This  chapter  will  propose  a  set  of  customer  satisfaction  (CS)  measures  for  NADEP 
North  Island's  F/A-18  Program,  based  on  the  information  presented  in  the  previous 
chapters.  It  will  also  discuss  implementation  of  the  proposed  CS  measures  and  how 
NADEP  NI  can  best  use  the  CS  information  they  provide.  The  proposed  set  of  customer 
satisfaction  measures  includes  the  following: 

-  a  single  point  of  contact  within  the  F/A-18  PMTO  for  customer  liaison 

-  a  customer  concern  tracking  system 

-  a  squadron  level  customer  satisfaction  survey 

-  a  CS  interview  survey  for  use  with  all  customer  levels 

-  a  customer  satisfaction  index  (CSI) 

A.  SINGLE  POINT  OF  CONTACT 

A  single  point  of  contact  (SPOC)  for  PMTO  customer  liaison  would  have  two 
important  benefits  for  the  PMTO  organization.  First,  it  would  make  it  easier  for 
customers  to  communicate  with  the  PMTO,  get  rapid  responses  to  questions  and  voice 
concerns.  Second,  it  would  serve  as  a  focal  point  for  all  PMTO  customer  satisfaction  data 
and  information.  This  would  facilitate  the  second  element  of  the  proposed  set  of  CS 
measures:  a  customer  concern  tracking  system. 

As  noted  in  Chapter  V,  questionnaire  feedback  from  squadrons  indicate  they  are 
very  satisfied  with  the  current  points  of  contact  they  have  with  NADEP  NI.  This  would 
appear  to  contradict  the  need  for  a  SPOC,  at  least  from  a  squadron  perspective.  In  reality, 
most  squadrons  do  not  often  contact  NADEP  NI  directly  for  MCAPP  scheduling  and 
related  information;  they  usually  go  through  their  wing  maintenance  officers  or  depot 
coordinators.  For  ISR  work  or  scheduling  PACE  inspections,  they  contact  the  local 
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NADEP  field  team  representative.  Complaints  may  be  recorded  on  ADRs  or  on  the 
depot's  customer  survey,  when  aircraft  are  returned  to  the  squadron.  Many  squadron 
concerns,  though,  are  probably  not  voiced  beyond  the  local  level.  Therefore,  these  are 
lost  improvement  opportunities  for  the  PMTO. 

As  stated  in  Chapter  III,  a  key  finding  of  the  National  Performance  Review's  report 
on  resolving  customer  complaints  was  the  importance  of  making  it  easy  for  customers  to 
complain.  (Gore  96)  The  PMTO  actively  encourages  customers  to  contact  the  PMTO. 
Telephone  and  fax  numbers  are  widely  distributed  to  customers  at  meetings  and  on  survey 
forms.  However,  customers  are  not  assured  of  getting  the  right  person  within  the  PMTO 
to  quickly  handle  their  questions  or  concerns.  Most  decisions  to  commit  resources  must 
be  approved  by  the  head  of  the  PMTO,  the  F/A-18  Product  Manager.  This  individual  has 
broad  responsibilities  and  is  understandably  not  always  immediately  available.  A  SPOC 
should  be  given  the  authority  to  approve  uncomplicated  requests  on  the  spot  when 
contacted  by  customers,  and  commit  resources  within  a  specified  scope  to  resolve 
problems:  this  would  significantly  improve  the  PMTO's  customer  interface. 

A  PMTO  CS  SPOC  could  also  notify  customers  of  schedule  changes  and  update 
the  LAST  bulletin  board  system  on  a  real  time  basis.  Currently,  the  LAST  system  is 
updated  once  every  two  weeks.  Some  customers  we  spoke  with  indicated  this  limited  the 
utility  of  the  system. 

This  CS  SPOC  function  could  be  tasked  to  an  existing  position  within  the  PMTO 
or  a  new  position  could  be  created  by  reassigning  duties  within  the  PMTO.  Alternately, 
various  PMTO  personnel  could  perform  the  CS  SPOC  function  on  a  rotating  basis.  All  of 
these  options  maintain  overall  manning  at  current  levels.  The  person(s)  filling  the  CS 
SPOC  billet  would  need  excellent  familiarity  with  NADEP's  mission,  processes  and 
customers,  training  in  CS  measurement  and  analytical  techniques,  familiarity  with  simple 
computer  database  programming,  and  strong  interpersonal  skills. 
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A  working  example  of  a  SPOC  for  all  customer  issues  is  NADEP  Cheny  Point's 
Customer  Liaison  Office  (CLO).  This  CLO  is  the  depot's  main  point  of  contact  for  all 
customer  matters,  including  routine  information  and  status  requests.  Cherry  Point's  CLO 
is  staffed  with  three  full-time  personnel,  and  is  the  SPOC  for  the  entire  NADEP,  not  just 
one  aircraft  program.  Because  of  the  size  of  the  F/A-18  program  and  the  PMTO,  a  SPOC 
dedicated  to  the  F/A-18  program  would  likely  provide  better  service  to  the  customer. 

B.  A  CUSTOMER  CONCERN  TRACKING  SYSTEM 

A  key  function  of  the  PMTO's  CS  single  point  of  contact  would  be  maintaining  a 
customer  concern  tracking  system.  NADEP's  F/A-18  PMTO  currently  collects  a  lot  of 
customer  satisfaction  data  by  various  means.  Much  of  the  potential  value  of  this 
information  goes  unrealized.  Not  all  CS  data  is  systematically  collected  and  analyzed,  and 
the  resulting  CS  information  made  available  to  all  personnel  within  the  PMTO. 

Specifically,  the  telephone  was  identified  by  PMTO  personnel  as  the  major  channel  for 
receiving  and  resolving  customer  concerns.  Customer  concerns  addressed  via  telephone 
to  PMTO  personnel,  and  the  associated  actions  taken,  are  not  recorded  in  any  systematic 
way.  An  improved  system  would  channel  all  customer  satisfaction  data,  whether  received 
by  telephone,  ADR,  survey,  interview,  etc.,  to  the  PMTO's  CS  SPOC.  The  SPOC  would 
be  responsible  for  collecting  and  recording  CS  data,  analyzing  it  and  then  disseminating 
CS  information  within  the  PMTO.  This  would  allow  a  better  picture  of  the  scope  and 
variety  of  customer  concerns  to  be  shared  by  all  members  of  the  PMTO  staff 

A  relational  database  is  the  simplest  way  to  record  all  customer  concerns, 
subsequently  analyze  the  data,  and  produce  reports.  Any  of  several  popular  commercial 
off-the-shelf  database  programs  would  work  for  this  relatively  small  database. 
Recommended  data  fields  to  record  for  each  customer  concern  include: 

customer  concern  file  number  (including  date  and  criticality  code) 
customer  (activity)  name 
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customer  POC 
customer  address 
customer  telephone  number 
customer  fax  number 

customer  type  (code  for  squadron,  wing,  TYCOM,  NAVAIR) 

BUNO  (if  concern  is  applicable  to  a  specific  aircraft) 
type  of  depot  work  (MCAPP,  ISR,  drive-in  mod,  etc.) 
type  of  concern  (special  request,  scheduled  delivery,  FOD,  etc.) 
details  of  concern 
date  received 

how  received  (ADR,  survey,  telephone,  interview,  etc.) 
date  action  assigned 
action  assignee 

date  of  initial  response  to  customer 
date  of  latest  interim  response  to  customer 
date  of  final  response  to  customer 
concern  status  (open  or  closed) 

C.  SQUADRON  LEVEL  CUSTOMER  SATISFACTION  SURVEY 

The  current  NADEP  NI  Quality  Process  Improvement  Questionnaire  (see  Chapter 
II,  Figure  2-6)  is  a  well-designed  and  useful  tool  to  measure  Squadron  level  customer 
satisfaction.  Based  on  the  CS  theory  material  presented  in  Chapter  IE,  some  parts  of  the 
questionnaire  can  be  improved;  the  response  data  can  also  be  better  used  by  the  depot. 
Also,  based  on  the  gaps  analysis  information  presented  in  Chapter  V  of  our  pretest 
questionnaire,  NADEP's  questionnaire  could  be  changed  to  focus  more  on  those  issues 
with  which  current  customers  are  dissatisfied. 
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This  survey  is  primarily  a  transaction  measure.  Its  purpose  is  to  gather  actionable 
data  concerning  a  particular  transaction  between  the  squadron  and  the  depot.  The  survey 
also  contains  some  questions  to  measure  relationships.  The  depot  needs  to  understand 
that  answers  or  grades  given  by  the  squadron  to  relationship  questions  will  be  greatly 
affected  by  their  experience  with  that  particular  transaction.  A  better  relationship 
measure  of  squadron  perceptions  would  be  performed  separately,  using  the  interview 
survey  discussed  in  the  next  section  of  this  chapter. 

Because  of  the  relatively  low  number  of  transactions,  and  their  high  cost,  the  depot 
should  get  survey  input  for  every  possible  transaction.  The  PMTO  CS  SPOC  may  wish  to 
follow-up  with  squadrons  from  whom  surveys  are  not  received.  Besides  tracking  down 
valuable  CS  data,  the  phone  call  is  another  opportunity  to  demonstrate  the  depot's 
commitment  to  CS.  The  survey  should  be  given  to  the  squadron  after  completing  MCAPP 
or  other  depot  level  maintenance,  and  after  completing  in-service  repair  (ISR)  work  or 
drive-in  modification  (DEM)  work  at  field  locations.  The  survey  should  always  be 
provided  with  a  point  of  contact  name  and  phone  number,  in  case  of  questions,  and  with  a 
self-addressed  stamped  envelope.  For  consistency  and  quality  of  response,  the  survey 
should  be  filled  out  by  the  squadron's  Maintenance/Material  Control  Officer  (MMCO).  By 
position,  this  one  individual  has  the  best  overall  knowledge  of  the  work  performed  by  the 
depot,  the  depot  coordination  function  within  the  squadron  and  the  impacts  of  schedule 
issues  on  operational  commitments. 

As  mentioned  in  the  SPOC  section  above,  if  the  survey  response  contains  any 
significant  customer  concerns,  the  concerns  should  be  entered  into  the  PMTO's  CS 
database  and  the  SPOC  should  follow-up  with  a  phone  call  to  the  squadron. 

Two  squadron  related  areas  stood  out  in  our  pretest  questionnaire  as  areas  where 
NADEP  NI  could  improve  customer  satisfaction.  These  were  consistency  of  PACE 
inspections  and  notification  by  NADEP  of  schedule  changes.  Other  areas  with  smaller 
performance-expectation  gaps  or  importance  ratings  include  the  following: 
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-  response  time  to  questions  or  requests  for  engineering  dispositions 

-  understanding  the  specific  work  that  was  performed  on  the  aircraft 

-  proper  documentation  in  aircraft  logbooks 

-  squadron  work  requests  over  and  above  the  MCAPP  specification 

-FOD 

-  impact  of  depot  maintenance  and  scheduling  on  the  squadron's  ability  to  operate 

It  is  interesting  to  note  that  basic  quality  issues  were  not  seen  as  dissatisfiers  by  the 

squadrons  on  our  pretest  questionnaire.  There  is  an  explanation  for  this,  beyond  the  fact 
that  the  quality  of  NADEP  NI's  work  is  indeed  very  high.  In  naval  aviation  maintenance, 
quality  is  really  "a  given."  Therefore,  most  squadrons  are  most  dissatisfied  with  the 
"supplemental"  aspects  of  depot  maintenance.  (Sloan  1994)  NADEP  NI's  squadron  level 
survey  must  retain  questions  dealing  with  basic  quality  issues  to  ensure  squadron 
satisfaction  is  maintained. 

Based  on  the  above.  Figure  6-1  is  a  proposed  revision  of  NADEP  NI's  Quality 
Process  Improvement  Questionnaire. 

D.  A  CS  INTERVIEW  SURVEY  FOR  USE  WITH  ALL  CUSTOMER  LEVELS 

In  order  to  measure  relationship  issues  at  the  squadron  level,  and  to  measure 
general  customer  satisfaction  at  the  other  customer  levels  (wing,  TYCOM,  NAVAIR), 
another  type  of  survey  is  needed.  Figure  6-2  is  a  proposed  CS  interview  survey  for  use 
with  all  customer  levels.  An  interview  survey  was  chosen  based  on  the  theoretical  CS 
information  presented  in  Chapter  III  and  the  experience  of  NADEP  Cherry  Point's 
Customer  Liaison  Office.  There  are  three  key  advantages  to  using  an  interview  survey. 
First,  the  response  rate  is  obviously  very  high.  Second,  the  interviewer  can  be  sure  they 
are  talking  to  the  correct  person  within  the  customer  activity.  If  necessary,  answers  can  be 
obtained  from  two  or  more  persons  during  the  interview.  Finally,  the  interview  format 
often  gathers  important  data  which  wouldn't  be  reflected  in  simple  answers 


Quality  Process  Improvement  Questionnaire 


T?’®  provides  NAVAL  AVIATION 

INUKIH  ISLAND  with  invaluable  information  from  the  F/A-15? 

iSprovemL  Iffoif  1^°'’  1°“  ^  “ntinaous  Ji^cess 


B  UNO -  NADEP  NI SEQ  NO.  (IF  KNOWN) 

DATE  AIRCRAFT  RETURNED  TO  SQUADRON _ 

TYPE  OF  DEPOT  WORK  (MCAPP,  ISR,  DRIVE-IN  MOD). 
LOCATION  WHERE  WORK  PERFORMFD 


N^DEP  north  ISLAND’S 
PERFORMANCE  IN  EACH  OF  THE  FOLLOWING  AREAS: 


A.  AIRCRAFT  CONDITION 

Overall  Workmanship - 

Paint  (lAW  NA  Ol-lA-509,  Appearance) - Z.'”“VZ 

Flight  Controls  (rigging,  security) - 

Power  Plants  (rigging,  security) - 

Interior  (FOD  free,  security) - 

Electrical  and  Avionics  (condition,  security) - 

Overall  Cleanliness - ... _ 

Logbooks  (completeness  and  accuracy  of  depot  entries) _ 

B.  SERVICE  (AS  APPLICABLE) 

Pre-Depot  Inspection  Process  (PACE/  PDM,  consistency) _ 

Depot  Availability  (wait  for  services) - 

Commumcations  fm  Depot  to  Squadron  (schedule  changes)  ---- 

Ease  of  Communicating  with  Depot - 

Schedule  Performance  (on  time  delivery) _ 

Rapid  Response  (engineering  dispositions,  requests) - 

Depot  Efforts  to  Minimize  Operational  Impact - 

Depot  Efforts  to  Accommodate  Special  Requests _ 

Depot  Efforts  to  Ensure  My  Understanding  of  Work  Performed 

C.  GENERAL  COMMENTS/RECOMMENDATIONS: 


LEGEND 

POOR  0 

MARGINAL  1 
FAIR  2 

VERY  GOOD  3 
EXCELLENT  4 


2  i  3 


Squadron  MMCO:. - -  Activity:  _  Phone:  - 

pjeasp  rqtum  ct^rqpleted  form  in  provided  stamped  pre-addressed  envelope  to: 

Officer  Naval  Al'iation  Depot,  Code  54200,  PO  Box  357058,  San  Diego,  CA  92135-7058 

Figure  «-I.  Proposed  F/A-18  PMTO  Quail.,  Pr„ee«  In.p™veu..„,  Ques,l„„„.ire 
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NADEP  North  Island  F/A-18  PMTO 
Customer  Satisfaction  Interview  Survey 


Customer  (activity)  name _  Interviewer _  Date  of  interview _ 

Customer  POC _  Telephone  number _  Fax  number _ 

Customer  mailing  address _ _ _ _ _ 

Customer  type:  _ squadron  _ wing  _ TYCOM  _ NAVAIR  _ other 

1.  What  NADEP  North  Island  F/A-18  products  or  services  do  you  use?  How  would  you  rate 
these  products  and  services  in  terms  of  schedule  performance  and  quality? 


Product  or  Service 

Sched.  Performance 

Oualitv  Grade  1 

(Example:  MCAPPJSR) 

Acceptable 

Unaccept. 

Poor 

Marginal 

m 

Very  Good 

Excellent 

wm 

Additional  Comments: 


2.  Are  NADEP  NTs  products  and  services  ideally  structured  to  meet  your  needs?  How  could  they 
be  better?  _ 


3.  Is  NADEP  NTs  F/A-18  PMTO  responsive  to  your  needs  and  concerns?  How  could  we 
improve?  _ _ _ _ _ 


4.  Do  you  feel  NADEP  NTs  F/A-18  PMTO  gives  you  value  for  your  money?  How  could  we 
improve?  _ _ _ 


5.  Would  you  consider  going  elsewhere  for  the  services  you  are  currently  receiving  from  NADEP 
NI?  If  so,  why?  _ 


Note  to  Interviewer:  customer  feedback  required?  _ Yes  _ No 

Figure  6-2.  Proposed  F/A-18  PMTO  Customer  Satisfaction  Interview  Survey 
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provided  on  a  survey  form.  The  main  drawback  of  this  type  of  survey  is  that  customers 
may  be  reluctant  to  criticize  an  activity  in  a  face-to-face  or  telephone  situation.  A  skilled 
interviewer  can  convince  the  customer  that  they  value  truly  candid  answers. 

Most  of  the  questions  on  this  survey  are  similar  to  the  depot  CS  survey  conducted 
twice  annually  by  NAVAIR  Code  6.0.  This  survey  should  be  administered  in-person,  or 
possibly  over  the  phone.  It  is  relatively  quick  to  complete.  The  questions  are  broad  in 
nature  so  they  can  be  used  for  a  variety  of  customers;  however,  they  are  designed  to  elicit 
specific  areas  of  dissatisfaction  from  whoever  is  being  interviewed. 

This  survey  would  be  administered  by  the  PMTO'S  CS  SPOC  during  visits  to 
customer  sites,  at  meetings  like  the  quarterly  F/A-18  Modification  Management  Meeting, 
which  provide  opportunities  to  talk  with  a  variety  of  customers,  or  over  the  phone. 
Customer  concern  data  would  be  entered  into  the  customer  concern  database. 

E.  A  CUSTOMER  SATISFACTION  INDEX 

The  fifth  and  final  element  of  the  proposed  set  of  CS  measures  for  NADEP  NT's 
F/A-18  PMTO  is  a  Customer  Satisfaction  Index  (CSI).  This  is  a  monthly  average  of  all 
quantifiable  data  points  fi'om  Quality  Process  Improvement  Questionnaires  and  Customer 
Satisfaction  Interview  Surveys  received  during  the  month.  The  scales  on  both  surveys  are 
identical  (Poor  -  0,  Marginal  - 1,  Fair  -  2,  Very  Good  -  3.  Excellent  -  4).  The  monthly 
average  is  plotted  on  a  4.0  scale. 

Though  it  is  similar  in  appearance,  this  CSI  differs  significantly  fi'om  the  GPA 
measure  currently  used  by  the  PMTO.  The  CSI  includes  CS  survey  data  from  several 
different  types  of  customers,  and  data  points  fi'om  both  transaction  and  relationship 
measures.  Because  it  incorporates  a  variety  of  measures  and  all  levels  of  the  PMTO's 
customers,  it  is  a  better  overall  measure.  However,  PMTO  managers  must  understand  the 
limitations  of  this,  or  any  CSI.  It  is  only  useful  for  measuring  an  overall  trend. 
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Understanding  which  program  areas  satisfy  customers,  and  which  areas  need 
improvement,  requires  a  detailed  analysis  of  the  survey  data. 

Figure  6-3  shows  a  sample  CSI  chart. 

F.  IMPLEMENTATION  ISSUES 

Several  things  need  to  be  considered  to  successfully  implement  the  CS 
measurement  system  described  above.  The  first  and  most  important  of  these  is  the 
decision  to  implement  the  system  folly  or  partially.  The  five  elements  proposed  above 
comprise  an  integrated  way  to  collect  and  analyze  CS  data,  and  make  CS  information 
available  about  all  four  of  the  PMTO's  customer  levels.  Some  of  the  elements  would  not 
be  effective  if  implemented  on  their  own,  or  without  certain  other  elements.  For  example, 
a  customer  concern  database  would  not  be  a  useful  resource  without  the  SPOC  function 
to  update  and  analyze  it.  Similarly,  continuing  to  do  CS  surveys  of  squadron  customers 
while  ignoring  other  PMTO  customer  levels  gives  a  distorted  view  of  how  well  the  PMTO 
is  accomplishing  its  mission.  Implementing  less  than  the  complete  set  of  CS  measures 
would  limit  what  the  PMTO  could  learn  about  its  complex  set  of  customer  requirements. 

The  two  key  issues  in  successfully  implementing  the  system  are  gaining  the  PMTO 
organization's  support  and  setting  up  the  CS  SPOC  position  within  the  PMTO.  Gaining 
PMTO's  support  is  not  a  trivial  matter.  It  involves  a  major  change  in  how  customer 
concerns  are  viewed.  Currently,  the  PMTO  is  justifiably  proud  of  the  low  number  of 
"complaints"  it  receives  (validated  ADR  discrepancies  and  negative  scores/comments  from 
squadron  surveys).  Management  based  on  customer  satisfaction  views  customer  feedback 
as  absolutely  vital  to  gauging  the  organization's  success  and  setting  its  direction.  This 
requires  actively  seeking  out  customer  concerns  and  acting  on  them.  To  collect  as  much 
CS  data  as  possible,  all  PMTO  personnel  need  to  appreciate  the  value  of  customer 
concerns  in  improving  the  quality  of  services  they  provide.  The 
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authors  believe  the  information  presented  in  this  thesis  and  the  references  present  a 
convincing  case  for  adopting  these  CS  measures;  similar  material  might  help  generate 
support  within  the  PMTO. 

Notwithstanding  the  above,  NADEP  NI  personnel  are  likely  to  be  supportive  of  an 
improved  CS  measurement  system.  The  depot's  experience  of  losing  work  to  the  Air 
Force  and  their  subsequent  dramatic  performance  improvement  is  still  fresh  in  the  minds  of 
most  NADEP  NI  personnel.  This  episode  occured  because  of  dissatisfied  customers.  The 
depot  has  done  a  lot  of  TQM  training  in  the  past  several  years,  so  most  NADEP  NI 
personnel  understand  the  concept  of  gauging  performance  from  a  customer  point  of  view 
There  has  also  been  a  lot  of  recent  press  given  to  CS  and  CS  measurement,  from  a  variety 
of  sources.  Therefore,  there  is  no  better  time  to  implement  this  system  than  the  present. 

Establishing  the  CS  SPOC  position  within  the  PMTO  is  a  significant  organizational 
change.  As  stated  above,  the  position  requires  a  certain  amount  of  decision  authority 
when  responding  to  customer  concerns.  The  nature  of  the  position  and  the  tasks  involved 
would  change  at  least  some  daily  routines  and  possibly  the  responsibilities  of  several  other 
PMTO  positions.  This  position  needs  to  be  implemented  with  sensitivity  to  the  needs  and 
desires  of  the  people  within  the  PMTO.  Some  options  for  staffing  this  position  have 
already  been  provided.  The  SPOC  billet  might  also  be  filled  by  one  of  the  military  officers 
assigned  to  the  PMTO.  This  would  lessen  the  impact  on  civilian  positions.  A  military 
officer  may  be  best  suited  to  interfacing  with  the  PMTO's  customers,  who  are  mostly 
military  officers  as  well. 

There  are  some  other  incidental  implementation  issues.  To  best  use  the  CS 
interview  survey,  the  PMTO  should  have  a  plan  for  completing  a  minimum  number  of 
these  from  each  customer  level  within  a  given  time  period.  For  example,  the  PMTO  may 
decide  that  they  need  responses  from  three  Navy  and  one  Marine  Corps  squadron,  one 
east  and  one  west  coast  wing,  two  from  TYCOMs  and  one  from  NAVAIR.  Based  on 
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travel  and  meeting  schedules  for  the  quarter,  the  PMTO  can  integrate  these  surveys  into 
their  workload. 

To  maintain  customer  support  in  filling  out  surveys  and  providing  honest  and 
thoughtful  feedback,  customers  must  feel  that  the  information  they  provide  is  being  put  to 
good  use.  PMTO  process  improvements  resulting  from  CS  measurement  need  to  be 
communicated  to  customers.  Hopefully,  most  of  these  improvements  will  be  obvious  to 
customers.  They  should  also  be  advertised  by  the  PMTO  at  meetings,  during  customer 
interviews,  in  NADEP's  promotional  material,  through  in-house  newsletters,  on  electronic 
bulletin  boards  (such  as  LAST)  and  on  the  NADEP's  web  site. 

In  time,  as  process  improvements  are  made  or  changes  occur  in  the  products  the 
PMTO  provides,  the  set  of  CS  measures  will  need  to  be  reevaluated.  Customer 
expectations  and  importance  ratings  of  different  attributes  will  change.  New  CS  attributes 
may  arise.  To  adapt  the  PMTO's  CS  measures,  it  is  necessary  to  perform  another  gaps 
analysis  survey,  along  the  lines  of  the  survey  shown  in  Chapter  Four.  The  PMTO's  CS 
SPOC  should  be  able  to  coordinate  this  effort,  following  the  methodology  used  in  this 
thesis.  This  should  be  done  when  the  environment  and  the  PMTO's  processes  change 
significantly.  In  the  PMTO's  current  environment,  with  major  changes  being  discussed  for 
F/A-18  depot  maintenance,  another  gaps  analysis  survey  should  certainly  be  done  in  the 
next  two  to  three  years. 

G.  INTEGRATING  CS  INFORMATION  INTO  DECISION  MAKING  AND 
REWARD  SYSTEMS 

An  important  aspect  of  emphasizing  customer  satisfaction  is  integrating  CS 
information  into  the  organization's  decision  making  and  reward  systems.  If  CS 
information  is  not  used  when  making  relevant  decisions,  or  worse,  ignored,  the 
organization  is  wasting  resources  in  collecting  and  analyzing  CS  data.  To  ensure  CS 
information  is  used,  it  should  be  made  widely  available  throughout  the  organization. 
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Because  people  will  work  towards  those  goals  on  which  their  incentive  systems  are 
focused,  CS  improvement  must  be  rewarded.  However,  it  is  not  a  simple  task  to  reward 
government  employees  based  on  customer  satisfaction.  For  example,  rewards  can't  be  tied 
to  the  number  of  complaints  received.  If  rewarded  for  a  low  number  of  complaints,  the 
organization's  personnel  may  discourage  important  CS  feedback.  Being  rewarded  for  a 
high  number  of  customer  complaints  is  similarly  counterproductive  and  might  encourage 
recording  extraneous  data.  A  better  measure  of  CS  performance  might  be  tracking  the 
number  of  customer  concerns  resolved  or  quantifying  improvements  in  terms  of  customer 
time  or  money  saved.  Linking  individual  or  small  group  rewards  directly  to  a  customer 
satisfaction  index  is  also  not  recommended.  Controlling  the  index  would  likely  become 
the  goal,  instead  of  satisfying  customers.  Rewards  for  a  larger  group  (e.g.  -  the  entire 
organization)  based  on  improvement  of  a  CS  index  may  be  workable,  but  they  dilute  the 
incentive  effect.  In  any  case,  any  objective  performance  measure  considered  must  be 
thoroughly  analyzed  to  understand  what  probable  incentive  effects  it  will  have  on 
personnel,  and  to  see  if  the  performance  measure  can  be  "gamed." 

An  effective  way  for  the  PMTO  to  incentivize  individual  and  small  group  CS 
improvement  efforts  is  by  incorporating  CS  responsibilities  and  goals  within  applicable 
employee  position  descriptions,  performance  plans  and  appraisals.  Management  must  then 
honestly  assess  the  individual's  contribution  toward  providing  better  customer  service,  as 
measured  by  the  customers.  The  depot  should  ask  key  customers  for  input  on  the 
performance  of  NADEP  customer  contact  employees  (SPOC,  field  team  supervisors). 
Though  subjective,  this  method  can  support  rewards  through  promotions,  step  increases, 
performance  bonuses,  and  other  employee  recognition. 

H.  EDUCATING  THE  CUSTOMER 

A  final  consideration  in  measuring  customer  satisfaction  is  educating  the  customer. 
This  thesis  draws  on  the  widely  accepted  concept  that  customer  satisfaction  is  related  to 


related  to  the  gap,  or  difference,  between  customer  expectations  and  the  customer’s 
perceptions  of  performance.  That  gap  can  be  closed,  and  customer  satisfaction  increased, 
in  two  ways:  improving  service  and  changing  customer  expectations.  Expectations  are 
changed  by  educating  customers  about  the  organization's  capabilities  and  constraints. 
Educated  customers  provide  another  important  benefit.  The  feedback  fi'om  an  educated 
customer  is  much  more  useful  for  process  improvement  than  that  provided  by  a  customer 
who  doesn't  understand  the  organization's  business. 

Customers  won't  normally  seek  out  detailed  information  on  a  service  provider's 
business.  Therefore,  the  burden  of  educating  the  customer  falls  upon  the  service 
provider.  Customers  can  learn  about  an  organization's  capabilities  and  constraints  in  the 
same  ways  used  to  publicize  improvements  resulting  from  CS  measures.  These  include 
PMTO  advertisements  at  meetings,  during  customer  interviews,  in  NADEP's  promotional 
material,  on  electronic  bulletin  boards  (such  as  LAST)  and  on  NADEP's  web  site. 
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vn.  CONCLUSIONS  ,  RECOMMENDATIONS,  AND  AREAS  FOR  FURTHER 

RESEARCH 


A.  CONCLUSIONS 

This  thesis  discussed  the  importance  of  CS  measurement  in  a  DoD  depot  program 
and  presented  a  methodology  to  build  a  tailored  set  of  customer  satisfaction  measures  for 
that  depot  program.  This  methodology  has  wide  applicability  to  other  NADEP  North 
Island  programs,  DoD  depots  and  government  organizations  in  general.  From  this 
research,  the  following  can  be  concluded; 

1.  Customer  Satisfaction  Measurement  is  a  Critical  Management 
Activity  for  DoD  Depots 

Correctly  measuring  CS  can  lead  to  more  efficient  and  effective  depot  operations. 
This  can  occur  through  reduced  costs  by  identifying  non-value  adding  tasks,  by  setting 
appropriate  service  levels  and  priorities,  and  also  by  increasing  a  depot's  customer  base. 
Depots  must  make  it  easy  for  their  customers  to  complain.  They  must  record  and  track 
key  CS  indicators  relevant  to  their  products  and  processes.  They  must  act  on  the 
information  generated  by  their  CS  measures  and  demonstrate  their  responsiveness  to 
customer  desires. 

2.  CS  must  be  Measured  by  a  Variety  of  Measures 

No  single  measure  or  measurement  technique  can  provide  all  the  CS  information 
that  a  depot  needs.  A  well  designed  CS  measurement  system  will  include  direct  and 
indirect  measures,  relationship  and  transaction  measures,  and  measures  of  supplemental 
factors.  Measuring  CS  is  a  complex  task  for  a  depot  because  the  depot  has  several  levels 
of  customers.  Different  customers  care  about  different  things,  and  measure  their  own 
satisfaction  differently.  CS  measures  should  include  all  the  depot's  products  and  services. 
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especially  those  with  a  high  bearing  on  CS,  such  as  field  teams.  All  CS  measures  chosen 
must  provide  actionable  data  and  merit  their  administrative  cost  to  the  depot  and  the 
customer. 

A  "Gaps  Model"  is  an  Appropriate  Tool  for  Measuring  CS  at  DoD 

Depots 

The  technique  of  comparing  customer  expectations  with  customer  perceptions  of 
performance,  combined  with  customer  rankings  of  attribute  importance,  is  well  established 
in  CS  literature  as  well  as  practical  application.  The  model's  ability  to  identify  significant 
CS  attributes,  and  to  facilitate  analysis  by  different  customer  groups  makes  it  particularly 
suitable  to  a  depot's  complex  CSM  requirements. 

4.  Customers  Need  to  be  Educated 

Customer  satisfaction  can  be  increased  in  two  ways;  improving  service  and 
changing  customer  expectations.  Establishing  reasonable  customer  expectations 
potentially  offers  gains  for  the  least  cost.  Expectations  are  changed  by  educating 
customers  about  the  depot's  capabilities  and  constraints.  The  depot  must  take  the 
initiative  to  do  this,  as  customers  won't  do  it  on  their  own.  Educated  customers  provide 
the  additional  benefit  of  giving  the  depot  better  process  improvement  feedback. 

B.  RECOMMENDATIONS 

Specific  CS  focus  areas  and  action  recommendations  based  on  responses  to  our 
pretest  questionnaire  are  provided  at  the  end  of  Chapter  Five  for  the  NADEP  Ml  F/A-18 
PMTO's  use.  Additional  source  data  has  also  been  provided  to  the  PMTO.  The  following 
are  our  general  recommendations  to  improve  the  PMTO's  CSM  system: 
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1.  Implement  the  Full  Set  of  CS  Measures  Proposed  in  Chapter  VI  of 
this  Thesis. 

The  current  CS  measures  focus  solely  on  the  squadron  level  customer,  and  only 
measure  certain  attributes.  The  PMTO  does  not  record  most  of  the  customer  satisfaction 
data  it  receives,  and  therefore  opportunities  to  improve  processes  may  be  missed.  The 
proposed  measures  tracks  and  records  CS  data  from  all  customer  levels,  and  are  based  on 
an  analysis  of  current  customer  satisfaction  and  attribute  importance  levels  for  each 
customer  level.  A  PMTO  single  point  of  contact  for  customer  concerns  will  make  it  easier 
for  customers  to  communicate  with  the  PMTO.  CS  information  should  be  integrated  into 
decision  making  and  employee  reward  systems,  and  shared  throughout  the  depot  for 
process  improvement.  Non-value  adding  CSM  activities  should  be  identified  and 
discontinued. 

2.  Continue  to  Establish  Closer  Ties  with  Customers 

Though  the  depot  has  several  customer  levels,  the  number  of  customers  at  most 
levels  is  very  small.  It  is  therefore  possible  for  the  depot  to  maintain  very  close  ties  to  its 
customers.  The  closer  these  tie,  the  more  likely  that  the  depot  will  understand  its 
customers’  concerns.  The  depot  should  make  a  strong  effort  to  develop  goodwill  and 
easy  communications  with  customers,  and  be  responsive  to  concerns  presented.  This  will 
encourage  customers  to  provide  the  depot  the  quality  CS  feedback  it  needs  to  improve 
and  survive.  Towards  this  end,  NADEP  NI  should  continue  initiatives  such  as  lobb5dng  to 
attend  Operational  Advisory  Group  (OAG)  meetings  and  other  forums  where  customer 
concerns  are  discussed.  The  depot  must  also  be  willing  to  share  information  with  its 
customers  on  capabilities,  schedules  and  performance  to  the  limits  of  available  electronic 
tools. 
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3.  Educate  Customers,  Particularly  Squadrons 

The  nature  of  the  work  that  NADEP  NT  performs  is  largely  invisible  to  the 
squadron  customer;  the  most  visible  aspect  is  the  aircraft’s  paint  job.  Squadrons  are  the 
depot  s  ultimate  customer,  and  have  a  large  voice  in  how  naval  aviation  resources  are 
used.  The  depot  should  take  the  initiative  to  ensure  these  customers  understand  the 
content  of  depot  maintenance  programs  and  what  value  they  add  for  the  squadron 
customer.  To  keep  these  customers  engaged,  the  depot  must  be  responsive  and  publicize 
improvements  resulting  from  CS  concerns. 

4.  Share  Information  on  the  Depot  Customer  Liaison  Function  with 
other  Depots 

OPNAV  Instruction  4790.2F  requires  NADEPs  to  maintain  a  Customer  Liaison 
Program.  It  was  obvious  during  our  research  that  different  NADEPs  do  this  differently. 
As  experience  and  understanding  grow  in  this  area,  some  techniques  will  prove  more 
effective  than  others.  DoD  depots,  while  sometimes  potential  competitors,  should 
maintain  dialogues  Avith  one  another  to  remain  on  the  cutting  edge  of  public  sector  CS 
initiatives. 

C.  AREAS  FOR  FURTHER  RESEARCH 

In  the  course  of  this  research,  many  ideas  surfaced  which  could  provide  fruitful 
areas  for  further  research.  One  idea  specifically  tied  to  this  thesis  is  an  investigation  of  the 
actual  costs  and  benefits  of  implementing  this  CS  measurement  system  at  NADEP  North 
Island.  Some  other  ideas  are; 
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-  Redesigning  and  combining  the  Aircraft  Discrepancy  Report  with  other  customer 
feedback  mechanisms  to  create  a  simpler  and  more  useful  form  for  quality  assurance  and 
CSM  purposes. 

-  Using  NALDA  data  to  measure  the  improvement  in  aircraft 
performance/reliability  as  a  result  of  depot  maintenance. 

-  Designing  and  evaluating  a  system  in  which  the  cost  to  customers  of  depot 
maintenance  reflects  the  actual  work  performed  on  their  aircraft. 

-  Developing  better  customer  satisfaction  incentive  structures  and  reward  systems 
for  depots  and  other  government  activities. 

-  Comparing  the  long-term  performance  of  activities  rewarding  CS  improvement 
with  activities  using  traditional  reward  systems. 
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